XPath explained

XPath
Paradigm:Query language
Year:1998
Developer:W3C
Latest Release Version:3.1
Influenced By:XSLT, XPointer
Influenced:XML Schema, XForms, JSONPath

XPath (XML Path Language) is an expression language designed to support the query or transformation of XML documents. It was defined by the World Wide Web Consortium (W3C) in 1999,[1] and can be used to compute values (e.g., strings, numbers, or Boolean values) from the content of an XML document. Support for XPath exists in applications that support XML, such as web browsers, and many programming languages.

Overview

The XPath language is based on a tree representation of the XML document, and provides the ability to navigate around the tree, selecting nodes by a variety of criteria.[2] [3] In popular use (though not in the official specification), an XPath expression is often referred to simply as "an XPath".

Originally motivated by a desire to provide a common syntax and behavior model between XPointer and XSLT, subsets of the XPath query language are used in other W3C specifications such as XML Schema, XForms and the Internationalization Tag Set (ITS).

XPath has been adopted by a number of XML processing libraries and tools, many of which also offer CSS Selectors, another W3C standard, as a simpler alternative to XPath.

Versions

There are several versions of XPath in use. XPath 1.0 was published in 1999, XPath 2.0 in 2007 (with a second edition in 2010), XPath 3.0 in 2014, and XPath 3.1 in 2017. However, XPath 1.0 is still the version that is most widely available.

The most notable change is that XPath 2.0 is built around the XQuery and XPath Data Model (XDM) that has a much richer type system. Every value is now a sequence (a single atomic value or node is regarded as a sequence of length one). XPath 1.0 node-sets are replaced by node sequences, which may be in any order.

To support richer type sets, XPath 2.0 offers a greatly expanded set of functions and operators.

XPath 2.0 is in fact a subset of XQuery 1.0. They share the same data model (XDM). It offers a for expression that is a cut-down version of the "FLWOR" expressions in XQuery. It is possible to describe the language by listing the parts of XQuery that it leaves out: the main examples are the query prolog, element and attribute constructors, the remainder of the "FLWOR" syntax, and the typeswitch expression.

Syntax and semantics (XPath 1.0)

The most important kind of expression in XPath is a location path. A location path consists of a sequence of location steps. Each location step has three components:

An XPath expression is evaluated with respect to a context node. An Axis Specifier such as 'child' or 'descendant' specifies the direction to navigate from the context node. The node test and the predicate are used to filter the nodes specified by the axis specifier: For example, the node test 'A' requires that all nodes navigated to must have label 'A'. A predicate can be used to specify that the selected nodes have certain properties, which are specified by XPath expressions themselves.

The XPath syntax comes in two flavors: the abbreviated syntax, is more compact and allows XPaths to be written and read easily using intuitive and, in many cases, familiar characters and constructs. The full syntax is more verbose, but allows for more options to be specified, and is more descriptive if read carefully.

Abbreviated syntax

The compact notation allows many defaults and abbreviations for common cases. Given source XML containing at least

the simplest XPath takes a form such as

that selects C elements that are children of B elements that are children of the A element that forms the outermost element of the XML document. The XPath syntax is designed to mimic URI (Uniform Resource Identifier) and Unix-style file path syntax.

More complex expressions can be constructed by specifying an axis other than the default 'child' axis, a node test other than a simple name, or predicates, which can be written in square brackets after any step. For example, the expression

selects the first child ('*[1]'), whatever its name, of every B element that itself is a child or other, deeper descendant ('//') of an A element that is a child of the current context node (the expression does not begin with a '/'). The predicate [1] binds more tightly than the / operator. To select the first node selected by the expression A//B/*, write (A//B/*)[1]. Note also, index values in XPath predicates (technically, 'proximity positions' of XPath node sets) start from 1, not 0 as common in languages like C and Java.

Expanded syntax

In the full, unabbreviated syntax, the two examples above would be written

Here, in each step of the XPath, the axis (e.g. child or descendant-or-self) is explicitly specified, followed by :: and then the node test, such as A or node in the examples above.

Here the same, but shorter:

Axis specifiers

Axis specifiers indicate navigation direction within the tree representation of the XML document. The axes available are:

Full syntax!scope="col"
Abbreviated syntaxNotes
is short for
is short for
is short for
is short for
is short for

As an example of using the attribute axis in abbreviated syntax, //a/@href selects the attribute called href in a elements anywhere in the document tree.The expression . (an abbreviation for self::node) is most commonly used within a predicate to refer to the currently selected node.For example, h3[.='See also'] selects an element called h3 in the current context, whose text content is See also.

Node tests

Node tests may consist of specific node names or more general expressions. In the case of an XML document in which the namespace prefix gs has been defined, //gs:enquiry will find all the enquiry elements in that namespace, and //gs:* will find all elements, regardless of local name, in that namespace.

Other node test formats are:

:finds an XML comment node, e.g. <nowiki><!-- Comment --></nowiki>
  • :finds a node of type text excluding any children, e.g. the hello in <k>hello<m> world</m></k>
  • :finds XML processing instructions such as . In this case, processing-instruction('php') would match.
  • :finds any node at all.
  • Predicates

    Predicates, written as expressions in square brackets, can be used to filter a node-set according to some condition. For example, a returns a node-set (all the a elements which are children of the context node), and keeps only those elements having an href attribute with the value help.php.

    There is no limit to the number of predicates in a step, and they need not be confined to the last step in an XPath. They can also be nested to any depth. Paths specified in predicates begin at the context of the current step (i.e. that of the immediately preceding node test) and do not alter that context. All predicates must be satisfied for a match to occur.

    When the value of the predicate is numeric, it is syntactic-sugar for comparing against the node's position in the node-set (as given by the function position). So p[1] is shorthand for and selects the first p element child, while p[last] is shorthand for and selects the last p child of the context node.

    In other cases, the value of the predicate is automatically converted to a Boolean. When the predicate evaluates to a node-set, the result is true when the node-set is . Thus p[@x] selects those p elements that have an attribute named x.

    A more complex example: the expression selects the value of the target attribute of the first a element among the children of the context node that has its href attribute set to help.php, provided the document's html top-level element also has a lang attribute set to en. The reference to an attribute of the top-level element in the first predicate affects neither the context of other predicates nor that of the location step itself.

    Predicate order is significant if predicates test the position of a node. Each predicate takes a node-set returns a (potentially) smaller node-set. So will find a match only if the first a child of the context node satisfies the condition @href='help.php', while will find the first a child that satisfies this condition.

    Functions and operators

    XPath 1.0 defines four data types: node-sets (sets of nodes with no intrinsic order), strings, numbers and Booleans.

    The available operators are:

    The function library includes:

    Some of the more commonly useful functions are detailed below.

    Node set functions

    :returns a number representing the position of this node in the sequence of nodes currently being processed (for example, the nodes selected by an xsl:for-each instruction in XSLT).
  • :returns the number of nodes in the node-set supplied as its argument.
  • String functions

    :converts any of the four XPath data types into a string according to built-in rules. If the value of the argument is a node-set, the function returns the string-value of the first node in document order, ignoring any further nodes.
  • :concatenates two or more strings
  • : returns true if s1 starts with s2
  • :returns true if s1 contains s2
  • :example: substring("ABCDEF",2,3) returns .
  • :example: substring-before("1999/04/01","/") returns 1999
  • :example: substring-after("1999/04/01","/") returns 04/01
  • :returns number of characters in string
  • :all leading and trailing whitespace is removed and any sequences of whitespace characters are replaced by a single space. This is very useful when the original XML may have been prettyprint formatted, which could make further string processing unreliable.
  • Boolean functions

    :negates any Boolean expression.
    :evaluates to true.
    :evaluates to false.

    Number functions

    :converts the string values of all the nodes found by the XPath argument into numbers, according to the built-in casting rules, then returns the sum of these numbers.

    Usage examples

    Expressions can be created inside predicates using the operators: <nowiki>=, !=, <=, <, >=</nowiki> and <nowiki>></nowiki>. Boolean expressions may be combined with brackets and the Boolean operators and and or as well as the not function described above. Numeric calculations can use <nowiki>*, +, -, div</nowiki> and <nowiki>mod</nowiki>. Strings can consist of any Unicode characters.

    selects items whose price attribute is greater than twice the numeric value of their discount attribute.

    Entire node-sets can be combined ('unioned') using the vertical bar character |. Node sets that meet one or more of several conditions can be found by combining the conditions inside a predicate with 'or'.

    v[x or y] | w[z] will return a single node-set consisting of all the v elements that have x or y child-elements, as well as all the w elements that have z child-elements, that were found in the current context.

    Syntax and semantics (XPath 2.0)

    See main article: XPath 2.0.

    Syntax and semantics (XPath 3)

    See main article: XPath 3.

    Examples

    Given a sample XML document en.wikipedia.org de.wikipedia.org fr.wikipedia.org pl.wikipedia.org es.wikipedia.org en.wiktionary.org fr.wiktionary.org vi.wiktionary.org tr.wiktionary.org es.wiktionary.org

    The XPath expression /Wikimedia/projects/project/@nameselects name attributes for all projects, and /Wikimedia//editionsselects all editions of all projects, and selects addresses of all English Wikimedia projects (text of all edition elements where language attribute is equal to English). And the following selects addresses of all Wikipedias (text of all edition elements that exist under project element with a name attribute of Wikipedia).

    Implementations

    Command-line tools

    C/C++

    Free Pascal

    Implementations for database engines

    Java

    The Javapackage has been part of Java standard edition since Java 5[7] via the Java API for XML Processing. Technically this is an XPath API rather than an XPath implementation, and it allows the programmer the ability to select a specific implementation that conforms to the interface.

    JavaScript

    .NET Framework

    Perl

    PHP

    Python

    Ruby

    Scheme

    SQL

    Tcl

    Use in schema languages

    XPath is increasingly used to express constraints in schema languages for XML.

    See also

    External links

    Notes and References

    1. Web site: XML and Semantic Web W3C Standards Timeline . 2012-02-04.
    2. XPath—Retrieving Nodes from an XML Document . Bergeron, Randy . SQL Server Magazine . 2000-10-31 . 2011-02-24 . https://web.archive.org/web/20100726204716/http://www.sqlmag.com/article/xml/xpath-151-retrieving-nodes-from-an-xml-document.aspx . 2010-07-26 . dead .
    3. Web site: Course: The XPath Language. Pierre Geneves. Pierre Geneves. 2012.
    4. Web site: XML Path Language (XPath) 3.0 . World Wide Web Consortium (W3C) . 2014-04-02 . 2021-07-16.
    5. Web site: Kay . Michael . What's new in 3.0 (XSLT/XPath/XQuery) (plus XML Schema 1.1) . XML Prague 2012 . 2012-02-10 . 2021-07-16.
    6. Web site: XML Path Language (XPath) 3.1 . World Wide Web Consortium (W3C) . 2017-03-21 . 2021-07-16.
    7. Web site: javax.xml.xpath (Java SE 10 & JDK 10) . Java® Platform, Standard Edition & Java Development Kit Version 10 API Specification . 2021-07-17 . Since: 1.5.
    8. Web site: System.Xml Namespace . Microsoft Docs . 2020-10-25 . 2021-07-16.
    9. Web site: How To Crawl A Web Page with Scrapy and Python 3 . 2017-11-24 . Duke . Justin . 2016-09-29 . . Selectors are patterns we can use to find one or more elements on a page so we can then work with the data within the element. scrapy supports either CSS selectors or XPath selectors..
    10. Web site: MySQL :: MySQL 5.1 Reference Manual :: 12.11 XML Functions . dev.mysql.com . 2016-04-06 . https://web.archive.org/web/20160406093208/http://dev.mysql.com/doc/refman/5.1/en/xml-functions.html . 2016-04-06 . unfit . 2021-07-17.
    11. Web site: xml2 . PostgreSQL Documentation . 2014-07-24 . 2021-07-16.
    12. Loewer . Jochen . tDOM – A fast XML/DOM/XPath package for Tcl written in C . Proceedings of First European TCL/Tk User Meeting . 2000 . 16 July 2021.