XML Path Language (XPath) 4.0 WG Review Draft

2.1.1 Grammar Notation

The grammar of XPath 4.0 is defined using a version of EBNF defined in A.1.1 Notation. The notation is based on the EBNF dialect used in the XML specification, with two notable additions derived from the Invisible XML grammar:

(A ++ ",") represents a sequence of one or more comma-separated occurrences of A.

(A ** ",") represents a sequence of zero or more comma-separated occurrences of A.

For example the following production rule indicates that an Expr consists of one or more occurrences of ExprSingle, separated by commas:

In principle any production can be used for the separator, but in practice this notation is only used in cases where the separator is a simple constant string.

EBNF grammar rules appear throughout the specification for ease of reference, and the entire grammar is summarized in A.1 EBNF. For XPath 4.0, the top-level production rule is XPath, representing an XPath expression.

2.1.2 Values

[Definition: In the data model, a value is always a sequence.]

[Definition: A sequence is an ordered collection of zero or more items.]

[Definition: An item is either an atomic item, a node, or a function item.]

[Definition: An atomic item is a value in the value space of an atomic type, as defined in [XML Schema 1.0] or [XML Schema 1.1].]

[Definition: A node is an instance of one of the node kinds defined in [TITLE OF DM40 SPEC, TITLE OF Node SECTION]^DM40.] Each node has a unique node identity, a typed value, and a string value. In addition, some nodes have a name. The typed value of a node is a sequence of zero or more atomic items. The string value of a node is a value of type xs:string. The name of a node is a value of type xs:QName.

[Definition: A function item is an item that can be called using a dynamic function call.]

Maps (see 4.13.1 Maps) and arrays (see 4.13.2 Arrays) are specific kinds of function items.

[Definition: A sequence containing exactly one item is called a singleton.] An item is identical to a singleton sequence containing that item. Sequences are never nested—for example, combining the values 1, (2, 3), and ( ) into a single sequence results in the sequence (1, 2, 3). [Definition: A sequence containing zero items is called an empty sequence.]

[Definition: The term XDM instance is used, synonymously with the term value, to denote an unconstrained sequence of items.]

2.1.3 Namespaces and QNames

[Definition: A namespace binding is a pair comprising a namespace prefix (which is either an xs:NCName or empty), and a namespace URI.]

Element nodes have a property called in-scope namespaces. [Definition: The in-scope namespaces property of an element node is a set of namespace bindings, each of which associates a namespace prefix with a URI.] For a given element, one namespace binding may have an empty prefix; the URI of this namespace binding is referred to as the default in-scope namespace for the element.

In [XML Path Language (XPath) Version 1.0], the in-scope namespaces of an element node are represented by a collection of namespace nodes arranged on a namespace axis. As of XPath 2.0, the namespace axis is deprecated and need not be supported by a host language. A host language that does not support the namespace axis need not represent namespace bindings in the form of nodes.

[Definition: The default in-scope namespace of an element node] is the namespace URI to which the empty prefix is bound in the element’s in-scope namespaces. In the absence of an explicit binding for the empty prefix, the default in-scope namespace is absent^DM.

[Definition: An expanded QName is a triple: its components are a prefix, a local name, and a namespace URI. In the case of a name in no namespace, the namespace URI and prefix are both absent. In the case of a name in the default namespace, the prefix is absent.] When comparing two expanded QNames, the prefixes are ignored: the local name parts must be equal under the Unicode codepoint collation (Section 5.3.1 Collations^FO), and the namespace URI parts must either both be absent, or must be equal under the Unicode codepoint collation.

In the XPath 4.0 grammar, QNames representing the names of elements, attributes, functions, variables, types, or other such constructs are written as instances of the grammatical production EQName.

The EQName production allows a QName to be written in one of three ways:

• local-name only (for example, invoice).

  A name written in this form has no prefix, and the rules for determining the namespace depend on the context in which the name appears: the various rules used are summarized in 2.1.4 Expanding Lexical QNames. This form is a lexical QName.

• prefix plus local-name (for example, my:invoice).

  In this case the prefix and local name of the QName are as written, and the namespace URI is inferred from the prefix by examining the statically known namespaces in the static context where the QName appears; the context must include a binding for the prefix. This form is a lexical QName.

• URI plus local-name (for example, Q{http://example.com/ns}invoice).

  In this case the local name and namespace URI are as written, and the prefix is absent. This way of writing a QName is context-free, which makes it particularly suitable for use in expressions that are generated by software. This form is a URIQualifiedName. If the BracedURILiteral has no content (for example, Q{}invoice) then the namespace URI of the QName is absent.

[Definition: A lexical QName is a name that conforms to the syntax of the QName production].

The namespace URI value in a URIQualifiedName is whitespace normalized according to the rules for the xs:anyURI type in Section 3.2.17 anyURI ^XS1-2 or Section 3.3.17 anyURI ^XS11-2. It is a static error [err:XQST0070] if the namespace URI for an EQName is http://www.w3.org/2000/xmlns/.

Here are some examples of EQNames:

• pi is a lexical QName without a namespace prefix.

• math:pi is a lexical QName with a namespace prefix.

• Q{http://www.w3.org/2005/xpath-functions/math}pi specifies the namespace URI using a BracedURILiteral; it is not a lexical QName.

This document uses the following namespace prefixes to represent the namespace URIs with which they are listed. Although these prefixes are used within this specification to refer to the corresponding namespaces, not all of these bindings will necessarily be present in the static context of every expression, and authors are free to use different prefixes for these namespaces, or to bind these prefixes to different namespaces.

• xs: http://www.w3.org/2001/XMLSchema

• fn: http://www.w3.org/2005/xpath-functions

• array: http://www.w3.org/2005/xpath-functions/array

• map: http://www.w3.org/2005/xpath-functions/map

• math: http://www.w3.org/2005/xpath-functions/math

• err: http://www.w3.org/2005/xqt-errors (see 2.4.2 Identifying and Reporting Errors).

• output: http://www.w3.org/2010/xslt-xquery-serialization

[Definition: Within this specification, the term URI refers to a Universal Resource Identifier as defined in [RFC3986] and extended in [RFC3987] with the new name IRI.] The term URI has been retained in preference to IRI to avoid introducing new names for concepts such as “Base URI” that are defined or referenced across the whole family of XML specifications.

2.1.4 Expanding Lexical QNames

When a lexical QName appearing in an XPath 4.0 expression is expanded, the rules are as follows.

If the name contains a colon, then the prefix (the substring before the colon) is used to establish the corresponding URI by reference to the statically known namespaces in the static context. If there is no binding for the prefix in the statically known namespaces then a static error is raised [err:XPST0081].

If the lexical QName contains no colon (and therefore no prefix), the namespace URI part of the QName is inferred in one of a number of ways, depending on the syntactic context in which the name appears. The various rules are defined below, and are referred to throughout the specification.

• [Definition: When an unprefixed lexical QName is expanded using the no-namespace rule, it is interpreted as having an absent namespace URI.]

• [Definition: When an unprefixed lexical QName is expanded using the default function namespace rule, it uses the default function namespace from the static context.]

• [Definition: When an unprefixed lexical QName is expanded using the default type namespace rule, it uses the default namespace for elements and types. If this is absent, the no-namespace rule is used. If the default namespace for elements and types has the special value ##any, then the lexical QName refers to a name in the namespace http://www.w3.org/2001/XMLSchema.]

• [Definition: When an unprefixed lexical QName is expanded using the default element namespace rule, then it uses the default namespace for elements and types. If this is absent, or if it takes the special value ##any, then the no-namespace rule is used.]

• [Definition: When an unprefixed lexical QName is expanded using the element name matching rule rule, then it uses the default namespace for elements and types. If this is absent, then it uses the no-namespace rule. But if it takes the special value ##any, then the name is taken as matching any expanded QName with the corresponding local part, regardless of namespace: that is, the unprefixed name local is interpreted as *:local.]

2.2.1 Static Context

[Definition: The static context of an expression is the information that is available during static analysis of the expression, prior to its evaluation.] This information can be used to decide whether the expression contains a static error.

The individual components of the static context are described below.

In XPath 4.0, the static context for an expression is largely defined by the host language, that is, by the calling environment that causes an XPath expression to be evaluated. Most of the static context components are constant throughout an expression; the only exception is in-scope variables. (There are constructs in the language, such as the ForExpr and LetExpr, that add additional variables to the static context of their subexpressions.)

Some components of the static context, but not all, also affect the dynamic semantics of expressions. For example, casting of a string such as "xbrl:xbrl" to an xs:QName might expand the prefix xbrl to the namespace URI http://www.xbrl.org/2003/instance using the statically known namespaces from the static context; since the input string "xbrl:xbrl" is in general not known until execution time (it might be read from a source document), this means that the values of the statically known namespaces must be available at execution time.

• [Definition: XPath 1.0 compatibility mode. This value is true if rules for backward compatibility with XPath Version 1.0 are in effect; otherwise it is false. ]

• [Definition: Statically known namespaces. This is a mapping from prefix to namespace URI that defines all the namespaces that are known during static processing of a given expression.]

  The URI value is whitespace normalized according to the rules for the xs:anyURI type in Section 3.2.17 anyURI ^XS1-2 or Section 3.3.17 anyURI ^XS11-2.

  Note the difference between in-scope namespaces, which is a dynamic property of an element node, and statically known namespaces, which is a static property of an expression.

• [Definition: Default namespace for elements and types. This is either a namespace URI, or the special value "##any", or absent^DM. This indicates how unprefixed QNames are interpreted when they appear in a position where an element name or type name is expected.]

  • If the value is set to a namespace URI, this namespace is used for any such unprefixed QName. The URI value is whitespace-normalized according to the rules for the xs:anyURI type in Section 3.2.17 anyURI ^XS1-2 or Section 3.3.17 anyURI ^XS11-2.

  • The special value "##any" indicates that:

    • When an unprefixed QName is used as a name test for selecting named elements in an axis step, the name test will match an element having the specified local name, in any namespace or none.

    • When an unprefixed QName is used in a context where a type name is expected (but not as a function name), the default namespace is the xs namespace, http://www.w3.org/2001/XMLSchema.

    • In any other context, an unprefixed QName represents a name in no namespace.

  • If the value is absent^DM, an unprefixed QName representing an element or type name is interpreted as being in no namespace.

• [Definition: Default function namespace. This is either a namespace URI, or absent^DM. The namespace URI, if present, is used for any unprefixed QName appearing in a position where a function name is expected.] The URI value is whitespace-normalized according to the rules for the xs:anyURI type in Section 3.2.17 anyURI ^XS1-2 or Section 3.3.17 anyURI ^XS11-2

  In its simplest form its value is simply a whitespace-normalized xs:anyURI value (most commonly, the URI http://www.w3.org/2005/xpath-functions) to be used as the default namespace for unprefixed function names. However, the use of a more complex algorithm is not precluded, for example an algorithm which searches multiple namespaces for a matching name.

• [Definition: In-scope schema definitions is a generic term for all the element declarations, attribute declarations, and schema type definitions that are in scope during static analysis of an expression.] It includes the following three parts:

  • [Definition: In-scope schema types. Each schema type definition is identified either by an expanded QName (for a named type) or by an implementation-dependent type identifier (for an anonymous type). The in-scope schema types include the predefined schema types described in 3.5 Schema Types. ]

  • [Definition: In-scope element declarations. Each element declaration is identified either by an expanded QName (for a top-level element declaration) or by an implementation-dependent element identifier (for a local element declaration). ] An element declaration includes information about the element’s substitution group affiliation.

    [Definition: Substitution groups are defined in Section 2.2.2.2 Element Substitution Group ^XS1-1 and Section 2.2.2.2 Element Substitution Group ^XS11-1. Informally, the substitution group headed by a given element (called the head element) consists of the set of elements that can be substituted for the head element without affecting the outcome of schema validation.]

  • [Definition: In-scope attribute declarations. Each attribute declaration is identified either by an expanded QName (for a top-level attribute declaration) or by an implementation-dependent attribute identifier (for a local attribute declaration). ]

• [Definition: In-scope variables. This is a mapping from expanded QNames to sequence types. It defines the set of variables that are available for reference within an expression. The expanded QName is the name of the variable, and the type is the static type of the variable.]

  An expression that binds a variable extends the in-scope variables, within the scope of the variable, with the variable and its type. Within the body of an inline function expression, the in-scope variables are extended by the names and types of the function parameters.

• [Definition: In-scope named item types. This is a mapping from expanded QNames to named item types.]

  [Definition: A named item type is an ItemType identified by an expanded QName.]

  Named item types serve two purposes:

  • They allow frequently used item types, especially complex item types such as record types, to be given simple names, to avoid repeating the definition every time it is used.

  • They allow the definition of recursive types, which are useful for describing recursive data structures such as lists and trees. For details see 3.2.8.3.1 Recursive Record Types.

  Certain named item types (typically item types used in the signatures of built-in functions) are always available in the static context. These are defined in Section C Built-in named record types^FO.

  Note:

  Named item types can be defined in a host language such as XQuery 4.0 and in XSLT 4.0, but not in XPath 4.0 itself. They are available in XPath only if the host language provides the ability to define them.

• [Definition: Statically known function definitions. This is a set of function definitions.]

  Function definitions are described in 2.2.1.1 Function Definitions.

• [Definition: Statically known collations. This is an implementation-defined mapping from URI to collation. It defines the names of the collations that are available for use in processing expressions.] [Definition: A collation is a specification of the manner in which strings and URIs are compared and, by extension, ordered. For a more complete definition of collation, see Section 5.3 Comparison of strings^FO.]

• [Definition: Static Base URI. This is an absolute URI, used to resolve relative URIs during static analysis. ] For example, it is used to resolve module location URIs in XQuery, and the URIs in xsl:import and xsl:include in XSLT. If E is a subexpression of F then the Static Base URI of E is the same as the Static Base URI of F. There are no constructs in XPath that require resolution of relative URI references during static analysis.

  Relative URI references are resolved as described in 2.5.6 Resolving a Relative URI Reference.

  At execution time, relative URIs supplied to functions such as fn:doc are resolved against the Executable Base URI, which may or may not be the same as the Static Base URI.

• [Definition: Statically known decimal formats. This is a mapping from QNames to decimal formats, with one default format that has no visible name, referred to as the unnamed decimal format. Each format is available for use when formatting numbers using the fn:format-number function.]

  Decimal formats are described in 2.2.1.2 Decimal Formats.

2.2.2 Dynamic Context

[Definition: The dynamic context of an expression is defined as information that is needed for the dynamic evaluation of an expression, beyond any information that is needed from the static context.] If evaluation of an expression relies on some part of the dynamic context that is absent^DM, a type error is raised [err:XPDY0002].

The individual components of the dynamic context are described below.

In general, the dynamic context for the outermost expression is supplied externally, often by some kind of application programming interface (API) allowing XPath 4.0 expressions to be invoked from a host language. Application Programming Interfaces are outside the scope of this specification. The dynamic context for inner subexpressions may be set by their containing expressions: for example in a mapping expression E1!E2 , the value of the focus (part of the dynamic context) for evaluation of E₂ is defined by the evaluation of E₁.

Some aspects of the dynamic context are outside the direct control of the query author; they are defined by the implementation, which may or may not allow them to be configured by users. An example is available documents, which is an abstraction for the set of XML documents that can be retrieved by URI from within an expression. In some environments this may be the entire contents of the web; in others it may be constrained to documents that satisfy particular security constraints; and in some environments the set of available documents might even be empty. These components of the dynamic context are generally treated as being constant for the duration of the execution.

Further rules governing the semantics of these components can be found in B.2 Dynamic Context Components.

The components of the dynamic context are listed below.

[Definition: The first three components of the dynamic context (context value, context position, and context size) are called the focus of the expression. ] The focus enables the processor to keep track of which items are being processed by the expression. If any component in the focus is defined, both the context value and context position are known.

[Definition: A fixed focus is a focus for an expression that is evaluated once, rather than being applied to a series of values; in a fixed focus, the context value is set to one specific value, the context position is 1, and the context size is 1.]

[Definition: A singleton focus is a fixed focus in which the context value is a singleton item.]. With a singleton focus, the context value is a single item, the context position is 1, and the context size is 1.

Certain language constructs, notably the path operator E1/E2 , the simple map operator E1!E2 , and the predicate E1[E2], create a new focus for the evaluation of a sub-expression. In these constructs, E₂ is evaluated once for each item in the sequence that results from evaluating E₁. Each time E₂ is evaluated, it is evaluated with a different focus. The focus for evaluating E₂ is referred to below as the inner focus, while the focus for evaluating E₁ is referred to as the outer focus. The inner focus is used only for the evaluation of E₂. Evaluation of E₁ continues with its original focus unchanged.

• [Definition: The context value is the value currently being processed.] In many cases (but not always), the context value will be a single item. [Definition: When the context value is a single item, it can also be referred to as the context item; when it is a single node, it can also be referred to as the context node.] The context value is returned by an expression consisting of a single dot (.). When an expression E1/E2 or E1[E2] is evaluated, each item in the sequence obtained by evaluating E1 becomes the context value in the inner focus for an evaluation of E₂.

• [Definition: The context position is the position of the context value within the series of values currently being processed.] It changes whenever the context value changes. When the focus is defined, the value of the context position is an integer greater than zero. The context position is returned by the expression fn:position(). When an expression E1/E2 or E1[E2] is evaluated, the context position in the inner focus for an evaluation of E2 is the position of the context value in the sequence obtained by evaluating E₁. The position of the first item in a sequence is always 1 (one). The context position is always less than or equal to the context size.

• [Definition: The context size is the number of values in the series of values currently being processed.] Its value is always an integer greater than zero. The context size is returned by the expression fn:last(). When an expression E1/E2 or E1[E2] is evaluated, the context size in the inner focus for an evaluation of E₂ is the number of items in the sequence obtained by evaluating E₁.

• [Definition: Variable values. This is a mapping from expanded QNames to values. It contains the same expanded QNames as the in-scope variables in the static context for the expression. The expanded QName is the name of the variable and the value is the dynamic value of the variable, which includes its dynamic type.]

• [Definition: Dynamically known function definitions. This is a set of function definitions. It includes the statically known function definitions as a subset, but may include other function definitions that are not known statically. ]

  The function definitions in the dynamic context are used primarily by the fn:function-lookup function.

  If two function definitions in the dynamically known function definitions have the same name, then their arity ranges must not overlap.

  Note:

  The reason for allowing named functions to be available dynamically beyond those that are available statically is primarily to allow for cases where the run-time execution environment is significantly different from the compile-time environment. This could happen, for example, if a stylesheet or query is compiled within a web server and then executed in the web browser. The fn:function-lookup function allows dynamic discovery of resources that were not available statically.

• [Definition: Current dateTime. This information represents an implementation-dependent point in time during the processing of an expression, and includes an explicit timezone. It can be retrieved by the fn:current-dateTime function. If called multiple times during the execution of an expression, this function always returns the same result.]

• [Definition: Implicit timezone. This is the timezone to be used when a date, time, or dateTime value that does not have a timezone is used in a comparison or arithmetic operation. The implicit timezone is an implementation-defined value of type xs:dayTimeDuration. See Section 3.2.7.3 Timezones ^XS1-2 or Section 3.3.7 dateTime ^XS11-2 for the range of valid values of a timezone.]

• [Definition: Executable Base URI. This is an absolute URI used to resolve relative URIs during the evaluation of expressions; it is used, for example, to resolve a relative URI supplied to the fn:doc or fn:unparsed-text functions. ]

  URIs are resolved as described in 2.5.6 Resolving a Relative URI Reference.

  The function fn:static-base-uri, despite its name, returns the value of the Executable Base URI.

  In many straightforward processing scenarios, the Executable Base URI in the dynamic context will be the same as the Static Base URI for the corresponding expression in the static context. There are situations, however, where they may differ:

  • Some processors may allow the static analysis of a query or stylesheet to take place on a development machine, while execution of the query or stylesheet happens on a test or production server. In this situation, resources needed during static analysis (such as other modules of the query or stylesheet) will be located on the development machine, by reference to the Static Base URI, while resources needed during execution (such as reference data files) will be located on the production machine, accessed via the Executable Base URI.

  • When the fn:static-base-uri function is called within the initializing expression of an optional parameter in a function declaration, it returns the executable base URI of the relevant function call. This allows a user-written function to accept two parameters: a required parameter containing a relative URI, and an optional parameter containing a base URI. The optional parameter can be given a default value of fn:static-base-uri(), allowing the code in the function body to resolve the relative URI against the executable base URI of the caller.

• [Definition: Default collation. This identifies one of the collations in statically known collations as the collation to be used by functions and operators for comparing and ordering values of type xs:string and xs:anyURI (and types derived from them) when no explicit collation is specified.]

  Note:

  Although the default collation is defined (in 4.0) as a property of the dynamic context, its value will in nearly all cases be known statically. The reason it is defined in the dynamic context is to allow a call on the fn:default-collation function to be used when defining the default value of an optional parameter to a user-defined function. In this situation, the actual value supplied for the parameter is taken from the dynamic context of the relevant function call.

• [Definition: Default language. This is the natural language used when creating human-readable output (for example, by the functions fn:format-date and fn:format-integer) if no other language is requested. The value is a language code as defined by the type xs:language.]

• [Definition: Default calendar. This is the calendar used when formatting dates in human-readable output (for example, by the functions fn:format-date and fn:format-dateTime) if no other calendar is requested. The value is a string.]

• [Definition: Default place. This is a geographical location used to identify the place where events happened (or will happen) when processing dates and times using functions such as fn:format-date, fn:format-dateTime, and fn:civil-timezone, if no other place is specified. It is used when translating timezone offsets to civil timezone names, and when using calendars where the translation from ISO dates/times to a local representation is dependent on geographical location. Possible representations of this information are an ISO country code or an Olson timezone name, but implementations are free to use other representations from which the above information can be derived. The only requirement is that it should uniquely identify a civil timezone, which means that country codes for countries with multiple timezones, such as the United States, are inadequate.]

• [Definition: Available documents. This is a mapping of strings to document nodes. Each string represents the absolute URI of a resource. The document node is the root of a tree that represents that resource using the data model. The document node is returned by the fn:doc function when applied to that URI.] The set of available documents may be empty.

• [Definition: Available text resources. This is a mapping of strings to text resources. Each string represents the absolute URI of a resource. The resource is returned by the fn:unparsed-text function when applied to that URI.] The set of available text resources may be empty.

• [Definition: Available binary resources. This is a mapping of strings to binary resources. Each string represents the absolute URI of a resource. The resource is returned by the fn:unparsed-binary function when applied to that URI.] The set of available binary resources may be empty.

• [Definition: Available collections. This is a mapping of strings to sequences of items. Each string represents the absolute URI of a resource. The sequence of items represents the result of the fn:collection function when that URI is supplied as the argument. ] The set of available collections may be empty.

  Ideally, for every document node D that is in the target of a mapping in available item collections, or that is the root of a tree containing such a node, the document-uri property of D should either be absent, or should be a URI U such that available documents contains a mapping from U to D.

  Note:

  That is to say, the document-uri property of nodes returned by the fn:collection function should be such that calling fn:doc with that URI returns the relevant node.

  It is not always possible to ensure this, especially in cases where dereferencing of document or collection URIs is configurable using configuration files or user-supplied resolver code.

• [Definition: Default collection. This is the sequence of items that would result from calling the fn:collection function with no arguments.] The value of default collection may be initialized by the implementation.

• [Definition: Available URI collections. This is a mapping of strings to sequences of URIs. The string represents the absolute URI of a resource which can be interpreted as an aggregation of a number of individual resources each of which has its own URI. The sequence of URIs represents the result of the fn:uri-collection function when that URI is supplied as the argument. ] There is no implication that the URIs in this sequence can be successfully dereferenced, or that the resources they refer to have any particular media type.

  Note:

  An implementation may maintain some consistent relationship between the available collections and the available URI collections, for example by ensuring that the result of fn:uri-collection(X)!fn:doc(.) is the same as the result of fn:collection(X). However, this is not required. The fn:uri-collection function is more general than fn:collection in that fn:collection allows access to nodes that might lack individual URIs, for example nodes corresponding to XML fragments stored in the rows of a relational database.

• [Definition: Default URI collection. This is the sequence of URIs that would result from calling the fn:uri-collection function with no arguments.] The value of default URI collection may be initialized by the implementation.

• [Definition: Environment variables. This is a mapping from names to values. Both the names and the values are strings. The names are compared using an implementation-defined collation, and are unique under this collation. The set of environment variables is implementation-defined and may be empty.]

  Note:

  A possible implementation is to provide the set of POSIX environment variables (or their equivalent on other operating systems) appropriate to the process in which the expression is evaluated.

2.3.1 Data Model Generation

The input data for an expression must be represented as one or more XDM instances. This process occurs outside the domain of XPath 4.0, which is why Figure 1 represents it in the external processing domain.

In many cases the input data might originate as XML. Here are some steps by which an XML document might be converted to an XDM instance:

1. A document may be parsed using an XML parser that generates an XML Information Set (see [XML Infoset]). The parsed document may then be validated against one or more schemas. This process, which is described in [XML Schema 1.0 Part 1] or [XML Schema 1.1 Part 1], results in an abstract information structure called the Post-Schema Validation Infoset (PSVI). If a document has no associated schema, its Information Set is preserved. (See DM1 in Figure 1)

2. The Information Set or PSVI may be transformed into an XDM instance by a process described in [XQuery and XPath Data Model (XDM) 4.0]. (See DM2 in Figure 1)

The above steps provide an example of how an XDM instance might be constructed. An XDM instance might also be constructed in some other way (see DM3 in Figure 1), for example it might be synthesized directly from a relational database, or derived by parsing a JSON text or a CSV file. Whatever the origin, XPath 4.0 is defined in terms of the data model, but it does not place any constraints on how XDM instances are constructed.

The remainder of this section is concerned with the common case where XML data is being processed.

[Definition: Each element node and attribute node in an XDM instance has a type annotation (described in Section 4.1 Schema Information^DM). The type annotation of a node is a reference to a schema type. ] The type-name of a node is the name of the type referenced by its type annotation (but note that the type annotation can be a reference to an anonymous type). If the XDM instance was derived from a validated XML document as described in Section 6.5.3 Construction from a PSVI^DM, the type annotations of the element and attribute nodes are derived from schema validation. XPath 4.0 does not provide a way to directly access the type annotation of an element or attribute node.

The value of an attribute is represented directly within the attribute node. An attribute node whose type is unknown (such as might occur in a schemaless document) is given the type annotation xs:untypedAtomic.

The value of an element is represented by the children of the element node, which may include text nodes and other element nodes. The type annotation of an element node indicates how the values in its child text nodes are to be interpreted. An element that has not been validated (such as might occur in a schemaless document) is annotated with the schema type xs:untyped. An element that has been validated and found to be partially valid is annotated with the schema type xs:anyType. If an element node is annotated as xs:untyped, all its descendant element nodes are also annotated as xs:untyped. However, if an element node is annotated as xs:anyType, some of its descendant element nodes may have a more specific type annotation.

2.3.2 Schema Import Processing

The in-scope schema definitions in the static context are provided by the host language (see step SI1 in Figure 1) and must satisfy the consistency constraints defined in 2.3.6 Consistency Constraints.

2.3.3 Expression Processing

XPath 4.0 defines two phases of processing called the static analysis phase and the dynamic evaluation phase (see Figure 1). During the static analysis phase, static errors, dynamic errors, or type errors may be raised. During the dynamic evaluation phase, only dynamic errors or type errors may be raised. These kinds of errors are defined in 2.4.1 Kinds of Errors.

Within each phase, an implementation is free to use any strategy or algorithm whose result conforms to the specifications in this document.

2.3.4 Input Sources

XPath 4.0 has a set of functions that provide access to XML documents (fn:doc, fn:doc-available), collections (fn:collection, fn:uri-collection), text files (fn:unparsed-text, fn:unparsed-text-lines, fn:unparsed-text-available), and environment variables (fn:environment-variable, fn:available-environment-variables). These functions are defined in Section 14.6 Functions giving access to external information^FO.

An expression can access input data either by calling one of these input functions or by referencing some part of the dynamic context that is initialized by the external environment, such as a variable or context value.

2.3.5 Serialization

[Definition: Serialization is the process of converting an XDM instance to a sequence of octets (step DM4 in Figure 1.), as described in [XSLT and XQuery Serialization 4.0].]

Serialization is outside the scope of the XPath specification, except to the extent that there is a function fn:serialize that enables serialization to be invoked.

2.3.6 Consistency Constraints

In order for XPath 4.0 to be well defined, the input XDM instances, the static context, and the dynamic context must be mutually consistent. The consistency constraints listed below are prerequisites for correct functioning of an XPath 4.0 implementation. Enforcement of these consistency constraints is beyond the scope of this specification. This specification does not define the result of an expression under any condition in which one or more of these constraints is not satisfied.

• For every node that has a type annotation, if that type annotation is found in the in-scope schema definitions (ISSD), then its definition in the ISSD must be compatible^DM with its definition in the schema^DM that was used to validate the node.

• Every element name, attribute name, or schema type name referenced in in-scope variables or statically known function definitions must be in the in-scope schema definitions, unless it is an element name referenced as part of an ElementTest or an attribute name referenced as part of an AttributeTest.

• Any reference to a global element, attribute, or type name in the in-scope schema definitions must have a corresponding element, attribute or type definition in the in-scope schema definitions.

• For each (variable, type) pair in in-scope variables and the corresponding (variable, value) pair in variable values such that the variable names are equal, the value must match the type, using the matching rules in 3.1.2 Sequence Type Matching.

• In the statically known namespaces, the prefix xml must not be bound to any namespace URI other than http://www.w3.org/XML/1998/namespace, and no prefix other than xml may be bound to this namespace URI. The prefix xmlns must not be bound to any namespace URI, and no prefix may be bound to the namespace URI http://www.w3.org/2000/xmlns/.

2.4.1 Kinds of Errors

As described in 2.3.3 Expression Processing, XPath 4.0 defines a static analysis phase, which does not depend on input data, and a dynamic evaluation phase, which does depend on input data. Errors may be raised during each phase.

[Definition: An error that can be detected during the static analysis phase, and is not a type error, is a static error.] A syntax error is an example of a static error.

[Definition: A dynamic error is an error that must be detected during the dynamic evaluation phase and may be detected during the static analysis phase.] Numeric overflow is an example of a dynamic error.

[Definition: A type error may be raised during the static analysis phase or the dynamic evaluation phase. During the static analysis phase, a type error occurs when the static type of an expression does not match the expected type of the context in which the expression occurs. During the dynamic evaluation phase, a type error occurs when the dynamic type of a value does not match the expected type of the context in which the value occurs.]

The outcome of the static analysis phase is either success or one or more type errors, static errors, or statically detected dynamic errors. The result of the dynamic evaluation phase is either a result value, a type error, or a dynamic error.

If more than one error is present, or if an error condition comes within the scope of more than one error defined in this specification, then any non-empty subset of these errors may be reported.

If an implementation can determine during the static analysis phase that an XPath expression, if evaluated, would necessarily raise a dynamic error or that an expression, if evaluated, would necessarily raise a type error, the implementation may (but is not required to) report that error during the static analysis phase.

An implementation can raise a dynamic error for an XPath expression statically only if the expression can never execute without raising that error, as in the following example:

error()

The following example contains a type error, which can be reported statically even if the implementation can not prove that the expression will actually be evaluated.

if (empty($arg))
then "cat" * 2
else 0

[Definition: In addition to static errors, dynamic errors, and type errors, an XPath 4.0 implementation may raise warnings, either during the static analysis phase or the dynamic evaluation phase. The circumstances in which warnings are raised, and the ways in which warnings are handled, are implementation-defined.]

In addition to the errors defined in this specification, an implementation may raise a dynamic error for a reason beyond the scope of this specification. For example, limitations may exist on the maximum numbers or sizes of various objects. An error must be raised if such a limitation is exceeded [err:XPDY0130].

2.4.2 Identifying and Reporting Errors

The errors defined in this specification are identified by QNames that have the form err:XPYYnnnn , where:

• err denotes the namespace for XPath and XQuery errors, http://www.w3.org/2005/xqt-errors. This binding of the namespace prefix err is used for convenience in this document, and is not normative.

• XP identifies the error as an XPath error (some errors, originally defined by XQuery and later added to XPath, use the code XQ instead).

• YY denotes the error category, using the following encoding:

  • ST denotes a static error.

  • DY denotes a dynamic error.

  • TY denotes a type error.

• nnnn is a unique numeric code.

2.4.3 Handling Dynamic Errors

Except as noted in this document, if any operand of an expression raises a dynamic error, the expression also raises a dynamic error. If an expression can validly return a value or raise a dynamic error, the implementation may choose to return the value or raise the dynamic error (see 2.4.4 Errors and Optimization). For example, the logical expression expr1 and expr2 may return the value false if either operand returns false, or may raise a dynamic error if either operand raises a dynamic error.

If more than one operand of an expression raises an error, the implementation may choose which error is raised by the expression. For example, in this expression:

($x div $y) + xs:decimal($z)

both the sub-expressions ($x div $y) and xs:decimal($z) may raise an error. The implementation may choose which error is raised by the + expression. Once one operand raises an error, the implementation is not required, but is permitted, to evaluate any other operands.

[Definition: In addition to its identifying QName, a dynamic error may also carry a descriptive string and one or more additional values called error values.] An implementation may provide a mechanism whereby an application-defined error handler can process error values and produce diagnostic messages. The host language may also provide error handling mechanisms.

A dynamic error may be raised by a system function or operator. For example, the div operator raises an error if its operands are xs:decimal values and its second operand is equal to zero. Errors raised by system functions and operators are defined in [XQuery and XPath Functions and Operators 4.0] or the host language.

A dynamic error can also be raised explicitly by calling the fn:error function, which always raises a dynamic error and never returns a value. This function is defined in Section 3.1.1 fn:error^FO. For example, the following function call raises a dynamic error, providing a QName that identifies the error, a descriptive string, and a diagnostic value (assuming that the prefix app is bound to a namespace containing application-defined error codes):

error( #app:err057, "Unexpected value", string($v) )

2.4.4 Errors and Optimization

Because different implementations may choose to evaluate or optimize an expression in different ways, certain aspects of raising dynamic errors are implementation-dependent, as described in this section.

An implementation is always free to evaluate the operands of an operator in any order.

In some cases, a processor can determine the result of an expression without accessing all the data that would be implied by the formal expression semantics. For example, the formal description of filter expressions suggests that $s[1] should be evaluated by examining all the items in sequence $s, and selecting all those that satisfy the predicate position()=1. In practice, many implementations will recognize that they can evaluate this expression by taking the first item in the sequence and then exiting. If $s is defined by an expression such as //book[author eq 'Berners-Lee'], then this strategy may avoid a complete scan of a large document and may therefore greatly improve performance. However, a consequence of this strategy is that a dynamic error or type error that would be detected if the expression semantics were followed literally might not be detected at all if the evaluation exits early. In this example, such an error might occur if there is a book element in the input data with more than one author subelement.

The extent to which a processor may optimize its access to data, at the cost of not raising errors, is defined by the following rules.

Consider an expression Q that has an operand (sub-expression) E. In general the value of E is a sequence. At an intermediate stage during evaluation of the sequence, some of its items will be known and others will be unknown. If, at such an intermediate stage of evaluation, a processor is able to establish that there are only two possible outcomes of evaluating Q, namely the value V or an error, then the processor may deliver the result V without evaluating further items in the operand E. For this purpose, two values are considered to represent the same outcome if their items are pairwise the same, where nodes are the same if they have the same identity, and values are the same if they are equal and have exactly the same type.

There is an exception to this rule: If a processor evaluates an operand E (wholly or in part), then it is required to establish that the actual value of the operand E does not violate any constraints on its cardinality. For example, the expression $e eq 0 results in a type error if the value of $e contains two or more items. A processor is not allowed to decide, after evaluating the first item in the value of $e and finding it equal to zero, that the only possible outcomes are the value true or a type error caused by the cardinality violation. It must establish that the value of $e contains no more than one item.

These rules apply to all the operands of an expression considered in combination: thus if an expression has two operands E1 and E2, it may be evaluated using any samples of the respective sequences that satisfy the above rules.

The rules cascade: if A is an operand of B and B is an operand of C, then the processor needs to evaluate only a sufficient sample of B to determine the value of C, and needs to evaluate only a sufficient sample of A to determine this sample of B.

The effect of these rules is that the processor is free to stop examining further items in a sequence as soon as it can establish that further items would not affect the result except possibly by causing an error. For example, the processor may return true as the result of the expression S1 = S2 as soon as it finds a pair of equal values from the two sequences.

Another consequence of these rules is that where none of the items in a sequence contributes to the result of an expression, the processor is not obliged to evaluate any part of the sequence. Again, however, the processor cannot dispense with a required cardinality check: if an empty sequence is not permitted in the relevant context, then the processor must ensure that the operand is not an empty sequence.

Examples:

• If an implementation can find (for example, by using an index) that at least one item returned by $expr1 in the following example has the value 47, it is allowed to return true as the result of the some expression, without searching for another item returned by $expr1 that would raise an error if it were evaluated.

  some $x in $expr1 satisfies $x = 47

• In the following example, if an implementation can find (for example, by using an index) the product element-nodes that have an id child with the value 47, it is allowed to return these nodes as the result of the path expression, without searching for another product node that would raise an error because it has an id child whose value is not an integer.

  //product[id = 47]

For a variety of reasons, including optimization, implementations may rewrite expressions into a different form. There are a number of rules that limit the extent of this freedom:

• Other than the raising or not raising of errors, the result of evaluating a rewritten expression must conform to the semantics defined in this specification for the original expression.

  Note:

  This allows an implementation to return a result in cases where the original expression would have raised an error, or to raise an error in cases where the original expression would have returned a result. The main cases where this is likely to arise in practice are (a) where a rewrite changes the order of evaluation, such that a subexpression causing an error is evaluated when the expression is written one way and is not evaluated when the expression is written a different way, and (b) where intermediate results of the evaluation cause overflow or other out-of-range conditions.

  Note:

  This rule does not mean that the result of the expression will always be the same in non-error cases as if it had not been rewritten, because there are many cases where the result of an expression is to some degree implementation-dependent or implementation-defined.

• The rules described in 2.4.5 Guarded Expressions ensure that for certain kinds of expression (for example conditional expressions), changing the order of evaluation of subexpressions does not result in dynamic errors that would not otherwise occur.

• Expressions must not be rewritten in such a way as to create or remove static errors. The static errors in this specification are defined for the original expression, and must be preserved if the expression is rewritten.

• As stated earlier, an expression must not be rewritten to dispense with a required cardinality check: for example, string-length(//title) must raise an error if the document contains more than one title element.

2.4.5 Guarded Expressions

[Definition: An expression E is said to be guarded by some governing condition C if evaluation of E is not allowed to fail with a dynamic error except when C applies.]

For example, in a conditional expression if (P) then T else F, the subexpression T is guarded by P, and the subexpression F is guarded by not(P). One way an implementation can satisfy this rule is by not evaluating T unless P is true, and likewise not evaluating F unless P is false. Another way of satisfying the rule is for the implementation to evaluate all the subexpressions, but to catch any errors that occur in a guarded subexpression so they are not propagated.

The existence of this rule enables errors to be prevented by writing expressions such as if ($y eq 0) then "N/A" else ($x div $y). This example will never fail with a divide-by-zero error because the else branch of the conditional is guarded.

Similarly, in the mapping expression E1!E2 , the subexpression E₂ is guarded by the existence of an item from E₁. This means, for example, that the expression (1 to $n)!doc('bad.xml') must not raise a dynamic error if $n is zero. The rule governing evaluation of guarded expressions is phrased so as not to disallow “loop-lifting” or “constant-folding” optimizations whose aim is to avoid repeated evaluation of a common subexpression; but such optimizations must not result in errors that would not otherwise occur.

The complete list of expressions that have guarded subexpressions is as follows:

• In a conditional expression (IfExpr) the then branch is guarded by the condition being true, and the else branch is guarded by the condition being false.

• In an and expression (AndExpr), the second operand is guarded by the value of the first operand being true.

• In an or expression (OrExpr), the second operand is guarded by the value of the first operand being false.

• In an otherwise expression (OtherwiseExpr), the second operand is guarded by the value of the first operand being an empty sequence.

• In a path expression of the form E1/E2 or E1//E2 , and in a mapping expression of the form E1!E2 , the right-hand operand E₂ is guarded by the existence of at least one item in the result of evaluating E₁.

  This rule applies even if E₂ does not reference the context value. For example, no dynamic error can be thrown by the expression (1 to $n)!doc('bad.xml') in the case where $n is zero.

• In a filter expression of the form E[P], the predicate P is guarded by the existence of at least one item in the result of evaluating E.

  This rule has the consequence that in a filter expression with multiple predicates, such as E[P1][P2], evaluation of P₂ must not raise a dynamic error unless P₁ returns true. This rule does not prevent reordering of predicates (for example, to take advantage of indexes), but it does require that any such reordering must not result in errors that would not otherwise occur.

• In a for expression (ForExpr) such as for $x in S return E , the expression E is guarded by the existence of an item bound to $x.

  This means that the expression for $x in 1 to $n return doc('bad.xml') must not raise a dynamic error in the case where $n is zero.

• In a quantified expression (QuantifiedExpr) such as some $x in S satisfies P , the expression P is guarded by the existence of an item bound to $x.

The fact that an expression is guarded does not remove the obligation to report static errors in the expression; nor does it remove the option to report statically detectable type errors.

2.4.6 Implausible Expressions

[Definition: Certain expressions, while not erroneous, are classified as being implausible, because they achieve no useful effect.]

An example of an implausible expression is @code/text(). This expression will always evaluate to an empty sequence, because attribute nodes cannot have text node children. The semantics of the expression are well defined, but it is likely that the user writing this expression intended something different: if they wanted to write an expression that evaluated to an empty sequence, there would be easier ways to write it.

Where an expression is classified (by rules in this specification) as being implausible, a processor may (but is not required to) raise a static error.

For reasons of backwards compatibility and interoperability, and to facilitate automatic generation of XPath 4.0 code, a processor must provide a mode of operation in which implausible expressions are not treated as static errors, but are evaluated with the defined semantics for the expression.

Some other examples of implausible expressions include:

• round(tokenize($input)). The result of fn:tokenize is a sequence of strings (xs:string*), while the required type for the first argument of fn:round is optional numeric (xs:numeric?). The expression can succeed only in the exceptional case where the result of fn:tokenize is an empty sequence, in which case the result of fn:round will also be an empty sequence; it is therefore highly likely that the expression was written in error.

• parse-csv($input)?column-names. The signature of the parse-csv function declares its return type as record(columns, rows). There is no field in this record named column-names, and therefore the lookup expression will always return an empty sequence. Again, there is no good reason that a user would write this, so it is likely that it was written in error.

2.5.1 Document Order

An ordering called document order is defined among all the nodes accessible during processing of a given expression, which may consist of one or more trees (documents or fragments). Document order is defined in Section 6.3 Document Order^DM, and its definition is repeated here for convenience. Document order is a total ordering, although the relative order of some nodes is implementation-dependent. [Definition: Informally, document order is the order in which nodes appear in the XML serialization of a document.] [Definition: Document order is stable, which means that the relative order of two nodes will not change during the processing of a given expression, even if this order is implementation-dependent.] [Definition: The node ordering that is the reverse of document order is called reverse document order.]

Within a tree, document order satisfies the following constraints:

1. The root node is the first node.

2. Every node occurs before all of its children and descendants.

3. Namespace nodes immediately follow the element node with which they are associated. The relative order of namespace nodes is stable but implementation-dependent.

4. Attribute nodes immediately follow the namespace nodes of the element node with which they are associated. The relative order of attribute nodes is stable but implementation-dependent.

5. The relative order of siblings is the order in which they occur in the children property of their parent node.

6. Children and descendants occur before following siblings.

The relative order of nodes in distinct trees is stable but implementation-dependent, subject to the following constraint: If any node in a given tree T1 is before any node in a different tree T2, then all nodes in tree T1 are before all nodes in tree T2.

2.5.2 Typed Value and String Value

Every node has a typed value and a string value, except for nodes whose value is absent^DM. [Definition: The typed value of a node is a sequence of atomic items and can be extracted by applying the Section 2.1.4 fn:data^FO function to the node.] [Definition: The string value of a node is a string and can be extracted by applying the Section 2.1.3 fn:string^FO function to the node.]

An implementation may store both the typed value and the string value of a node, or it may store only one of these and derive the other as needed. The string value of a node must be a valid lexical representation of the typed value of the node, but the node is not required to preserve the string representation from the original source document. For example, if the typed value of a node is the xs:integer value 30, its string value might be "30" or "0030".

The typed value, string value, and type annotation of a node are closely related. If the node was created by mapping from an Infoset or PSVI, the relationships among these properties are defined by rules in Section 4.1 Schema Information^DM.

The relationship between typed value and string value for various kinds of nodes is summarized and illustrated by examples below.

1. For text and document nodes, the typed value of the node is the same as its string value, as an instance of the type xs:untypedAtomic. The string value of a document node is formed by concatenating the string values of all its descendant text nodes, in document order.

2. The typed value of a comment, namespace, or processing instruction node is the same as its string value. It is an instance of the type xs:string.

3. The typed value of an attribute node with the type annotation xs:anySimpleType or xs:untypedAtomic is the same as its string value, as an instance of xs:untypedAtomic. The typed value of an attribute node with any other type annotation is derived from its string value and type annotation using the lexical-to-value-space mapping defined in [XML Schema 1.0] or [XML Schema 1.1] Part 2 for the relevant type.

   Example: A1 is an attribute having string value "3.14E-2" and type annotation xs:double. The typed value of A1 is the xs:double value whose lexical representation is 3.14E-2.

   Example: A2 is an attribute with type annotation xs:IDREFS, which is a list datatype whose item type is the atomic datatype xs:IDREF. Its string value is "bar baz faz". The typed value of A2 is a sequence of three atomic items ("bar", "baz"", "faz""), each of type xs:IDREF. The typed value of a node is never treated as an instance of a named list type. Instead, if the type annotation of a node is a list type (such as xs:IDREFS), its typed value is treated as a sequence of the generalized atomic type from which it is derived (such as xs:IDREF).

4. For an element node, the relationship between typed value and string value depends on the node’s type annotation, as follows:

   1. If the type annotation is xs:untyped or xs:anySimpleType or denotes a complex type with mixed content (including xs:anyType), then the typed value of the node is equal to its string value, as an instance of xs:untypedAtomic. However, if the nilled property of the node is true, then its typed value is the empty sequence.

      Example: E1 is an element node having type annotation xs:untyped and string value "1999-05-31". The typed value of E1 is "1999-05-31", as an instance of xs:untypedAtomic.

      Example: E2 is an element node with the type annotation formula, which is a complex type with mixed content. The content of E2 consists of the character H, a child element named subscript with string value "2", and the character O. The typed value of E2 is "H2O" as an instance of xs:untypedAtomic.

   2. If the type annotation denotes a simple type or a complex type with simple content, then the typed value of the node is derived from its string value and its type annotation in a way that is consistent with schema validation. However, if the nilled property of the node is true, then its typed value is the empty sequence.

      Example: E3 is an element node with the type annotation cost, which is a complex type that has several attributes and a simple content type of xs:decimal. The string value of E3 is "74.95". The typed value of E3 is 74.95, as an instance of xs:decimal.

      Example: E4 is an element node with the type annotation hatsizelist, which is a simple type derived from the atomic type hatsize, which in turn is derived from xs:integer. The string value of E4 is "7 8 9". The typed value of E4 is a sequence of three values (7, 8, 9), each of type hatsize.

      Example: E5 is an element node with the type annotation my:integer-or-string which is a union type with member types xs:integer and xs:string. The string value of E5 is "47". The typed value of E5 is 47 as a xs:integer, since xs:integer is the member type that validated the content of E5. In general, when the type annotation of a node is a union type, the typed value of the node will be an instance of one of the member types of the union.

      Note:

      If an implementation stores only the string value of a node, and the type annotation of the node is a union type, the implementation must be able to deliver the typed value of the node as an instance of the appropriate member type.

   3. If the type annotation denotes a complex type with empty content, then the typed value of the node is the empty sequence and its string value is the zero-length string.

   4. If the type annotation denotes a complex type with element-only content, then the typed value of the node is absent^DM. The fn:data function raises a type error [err:FOTY0012]^FO40 when applied to such a node. The string value of such a node is equal to the concatenated string values of all its text node descendants, in document order.

      Example: E6 is an element node with the type annotation weather, which is a complex type whose content type specifies element-only. E6 has two child elements named temperature and precipitation. The typed value of E6 is absent^DM, and the fn:data function applied to E6 raises an error.

2.5.3 Atomization

The semantics of some XPath 4.0 operators depend on a process called atomization. Atomization is applied to a value when the value is used in a context in which a sequence of atomic items is required. The result of atomization is either a sequence of atomic items or a type error [err:FOTY0012]^FO40. [Definition: Atomization of a sequence is defined as the result of invoking the fn:data function, as defined in Section 2.1.4 fn:data^FO.]

The semantics of fn:data are repeated here for convenience. The result of fn:data is the sequence of atomic items produced by applying the following rules to each item in the input sequence:

• If the item is an atomic item, it is returned.

• If the item is a node, its typed value is returned (a type error [err:FOTY0012]^FO40 is raised if the node has no typed value.)

• If the item is a function item (other than an array) or map a type error [err:FOTY0013]^FO40 is raised.

• If the item is an array $a, atomization is defined as $a?* ! fn:data(.), which is equivalent to atomizing the members of the array.

  Note:

  This definition recursively atomizes members that are arrays. Hence, the result of atomizing the array [ [ 1, 2, 3 ], [ 4, 5, 6 ] ] is the sequence (1, 2, 3, 4, 5, 6).

Atomization is used in processing the following types of expressions:

• Arithmetic expressions

• Comparison expressions

• Function calls and returns

• Cast expressions

2.5.4 Effective Boolean Value

Under certain circumstances (listed below), it is necessary to find the effective boolean value of a value. [Definition: The effective boolean value of a value is defined as the result of applying the fn:boolean function to the value, as defined in Section 8.3.1 fn:boolean^FO.]

The dynamic semantics of fn:boolean are repeated here for convenience:

1. If its operand is an empty sequence, fn:boolean returns false.

2. If its operand is a sequence whose first item is a node, fn:boolean returns true.

3. If its operand is a singleton value of type xs:boolean or derived from xs:boolean, fn:boolean returns the value of its operand unchanged.

4. If its operand is a singleton value of type xs:string, xs:anyURI, xs:untypedAtomic, or a type derived from one of these, fn:boolean returns false if the operand value has zero length; otherwise it returns true.

5. If its operand is a singleton value of any numeric type or derived from a numeric type, fn:boolean returns false if the operand value is NaN or is numerically equal to zero; otherwise it returns true.

6. In all other cases, fn:boolean raises a type error [err:FORG0006]^FO40.

   Note:

   For instance, fn:boolean raises a type error if the operand is a function, a map, or an array.

The effective boolean value of a sequence is computed implicitly during processing of the following types of expressions:

• Logical expressions (and, or)

• The fn:not function

• Certain types of predicates, such as a[b]

• Conditional expressions (if)

• Quantified expressions (some, every)

• General comparisons, in XPath 1.0 compatibility mode.

2.5.5 URI Literals

XPath 4.0 requires a statically known, valid URI in a BracedURILiteral. An implementation may raise a static error [err:XQST0046] if the value of a Braced URI Literal is of nonzero length and is neither an absolute URI nor a relative URI.

Whitespace is normalized using the whitespace normalization rules of fn:normalize-space. If the result of whitespace normalization contains only whitespace, the corresponding URI consists of the empty string.

A Braced URI Literal or URI Literal is not subjected to percent-encoding or decoding as defined in [RFC3986].

2.5.6 Resolving a Relative URI Reference

[Definition: To resolve a relative URI $rel against a base URI $base is to expand it to an absolute URI, as if by calling the function fn:resolve-uri($rel, $base).] During static analysis, the base URI is the Static Base URI. During dynamic evaluation, the base URI used to resolve a relative URI reference depends on the semantics of the expression.

Any process that attempts to resolve a URI against a base URI, or to dereference the URI, may apply percent-encoding or decoding as defined in the relevant RFCs.

3.1.1 Examples of Sequence Types

Here are some examples of sequence types that might be used in XPath 4.0:

• xs:date refers to the built-in atomic schema type named xs:date

• attribute()? refers to an optional attribute node

• element() refers to any element node

• element(po:shipto, po:address) refers to an element node that has the name po:shipto and has the type annotation po:address (or a schema type derived from po:address)

• element(*, po:address) refers to an element node of any name that has the type annotation po:address (or a type derived from po:address)

• element(customer) refers to an element node named customer with any type annotation

• schema-element(customer) refers to an element node whose name is customer (or is in the substitution group headed by customer) and whose type annotation matches the schema type declared for a customer element in the in-scope element declarations

• node()* refers to a sequence of zero or more nodes of any kind

• item()+ refers to a sequence of one or more items

• function(*) refers to any function item, regardless of arity or type

• function(node()) as xs:string* refers to a function item that takes a single argument whose value is a single node, and returns a sequence of zero or more xs:string values

• (fn(node()) as xs:string)* refers to a sequence of zero or more function items, each of which takes a single argument whose value is a single node, and returns as its result a single xs:string value

3.1.2 Sequence Type Matching

[Definition: SequenceType matching compares a value with an expected sequence type. ] For example, an instance of expression returns true if a given value matches a given sequence type, and false if it does not.

An XPath 4.0 implementation must be able to determine relationships among the types in type annotations in an XDM instance and the types in the in-scope schema definitions (ISSD).

[Definition: The use of a value that has a dynamic type that is a subtype of the expected type is known as subtype substitution.] Subtype substitution does not change the actual type of a value. For example, if an xs:integer value is used where an xs:decimal value is expected, the value retains its type as xs:integer.

The rules for SequenceType matching are given below, with examples (the examples are for purposes of illustration, and do not cover all possible cases).

• The sequence type empty-sequence() matches a value that is the empty sequence.

• An ItemType with no OccurrenceIndicator matches any value that contains exactly one item if the ItemType matches that item (see 3.2 Item Types).

• An ItemType with an OccurrenceIndicator matches a value if the number of items in the value matches the OccurrenceIndicator and the ItemType matches each of the items in the value.

An OccurrenceIndicator specifies the number of items in a sequence, as follows:

• ? matches zero or one items

• * matches zero or more items

• + matches one or more items

As a consequence of these rules, any sequence type whose OccurrenceIndicator is * or ? matches a value that is an empty sequence.

3.1.3 Schema Type Relationships

Some item types are defined in terms of schema types, and the matching rules for such item types depend on the rules defining relationships between schema types in the XSD specification.

[Definition: A schema type S₁ is said to derive from schema type S₂ if any of the following conditions is true:

• S₁ is the same type as S₂.

• S₂ is the base type of S₁.

• S₂ is a pure union type of which S₁ is a member type.

• There is a schema type M such that S₁ derives from M and M derives from S₂.

]

3.2.1 General item types

• item() matches any single item.

  For example, item() matches the atomic item 1, the element <a/>, or the function fn:concat#3.

• A ChoiceItemType lists a number of alternative item types in parentheses, separated by "|". An item matches a ChoiceItemType it if matches any of the alternatives.

  For example, (map(*) | array(*)) matches any item that is a map or an array.

  Note:

  If there is only one alternative, the ChoiceItemType designates the same item type as the ItemType that is in parentheses. A singleton choice (that is, a parenthesized item type) is used primarily when defining nested item types in a function signature. For example, a sequence of functions that each return a single boolean might be denoted (fn() as xs:boolean)*. In this example the parentheses are needed to indicate where the occurrence indicator belongs.

3.2.2 Atomic Types

Atomic types in the XPath 4.0 type system correspond directly to atomic types as defined in the [XML Schema 1.0] or [XML Schema 1.1] type system.

Atomic types are either built-in atomic types such as xs:integer, or user-defined atomic types imported from a schema. Atomic types are identified by a QName: see 2.1.3 Namespaces and QNames.

[Definition: A generalized atomic type is an item type whose instances are all atomic items. Generalized atomic types include (a) atomic types, either built-in (for example xs:integer) or imported from a schema, (b) pure union types, either built-in (xs:numeric and xs:error) or imported from a schema, (c) choice item types if their alternatives are all generalized atomic types, and (d) enumeration types. ].

A generalized atomic type may be designated by an ItemType in any of the following ways:

• Using the QName of a type in the in-scope schema definitions that is an atomic type or a pure union type.

• Using a QName that identifies a named item type that resolves to a generalized atomic type.

• Using a ChoiceItemType where every alternative is itself a generalized atomic type.

• Using an EnumerationType as described below.

An atomic item A matches the generalized atomic type GAT if the type annotation of A derives from GAT.

Example: The ItemType xs:decimal matches any value of type xs:decimal. It also matches any value of type shoesize, if shoesize is an atomic type derived by restriction from xs:decimal.

Example: Suppose ItemType dress-size is a union type that allows either xs:decimal values for numeric sizes (for example: 4, 6, 10, 12), or one of an enumerated set of xs:strings (for example: small, medium, large). The ItemType dress-size matches any of these values.

3.2.3 Union Types

Union types, as defined in XSD, are a variety of simple types. The membership of a union type in XSD may include list types as well as atomic types and other union types.

[Definition: A pure union type is a simple type that satisfies the following constraints: (a) {variety}^XS11-1 is union, (b) the {facets}^XS11-1 property is empty, (c) no type in the transitive membership of the union type has {variety}^XS11-1 list, and (d) no type in the transitive membership of the union type is a type with {variety}^XS11-1 union having a non-empty {facets}^XS11-1 property].

[Definition: The type xs:numeric is defined as a union type with member types xs:double, xs:float, and xs:decimal. An item that is an instance of any of these types is referred to as a numeric value, and a type that is a subtype of xs:numeric is referred to as a numeric type.]

3.2.4 Namespace-sensitive Types

[Definition: The namespace-sensitive types are xs:QName, xs:NOTATION, types derived by restriction from xs:QName or xs:NOTATION, list types that have a namespace-sensitive item type, and union types with a namespace-sensitive type in their transitive membership.]

It is not possible to preserve the type of a namespace-sensitive value without also preserving the namespace binding that defines the meaning of each namespace prefix used in the value. Therefore, XPath 4.0 defines some error conditions that occur only with namespace-sensitive values. For instance, casting to a namespace-sensitive type raises a type error [err:FONS0004]^FO40 if the namespace bindings for the result cannot be determined.

3.2.5 Choice Item Types

[Definition: A choice item type defines an item type that is the union of a number of alternatives. For example the type (xs:hexBinary | xs:base64Binary) defines the union of these two primitive atomic types, while the type (map(*) | array(*)) matches any item that is either a map or an array.]

An item matches a ChoiceItemType if it matches any of the alternatives listed within the parentheses.

For example, the type (xs:NCName | enum("")) matches any value that is either an instance of xs:NCName, or a zero-length string. This might be a suitable type for a variable that holds a namespace prefix.

If all the alternatives are generalized atomic types then the choice item type is itself a generalized atomic type, which means, for example, that it can be used as the target of a cast expression.

3.2.6 Enumeration Types

[Definition: An EnumerationType accepts a fixed set of string values.]

An enumeration type has a value space consisting of a set of xs:string values. When matching strings against an enumeration type, strings are always compared using the Unicode codepoint collation.

For example, if an argument of a function declares the required type as enum("red", "green", "blue"), then the string "green" is accepted, while "yellow" is rejected with a type error.

Technically, enumeration types are defined as follows:

• [Definition: An enumeration type with a single enumerated value (such as enum("red")) is an anonymous atomic type derived from xs:string by restriction using an enumeration facet that permits only the value "red". This is referred to as a singleton enumeration type.] It is equivalent to the XSD-defined type:

  <xs:simpleType>
    <xs:restriction base="xs:string">
      <xs:enumeration value="red"/>
    </xs:restriction>
  </xs:simpleType>

• Two singleton enumeration types are the same type if and only if they have the same (single) enumerated value, as determined using the Unicode codepoint collation.

• An enumeration type with multiple enumerated values is a union of singleton enumeration types, so enum("red", "green", "blue") is equivalent to (enum("red") | enum("green") | enum("blue")).

• In consequence, an enumeration type T is a subtype of an enumeration type U if the enumerated values of T are a subset of the enumerated values of U: see 3.3.2 Subtypes of Item Types.

An enumeration type is thus a generalized atomic type.

It follows from these rules that an atomic item will only satisfy an instance of test if it has the correct type annotation, and this typically requires an explicit cast. So the expression "red" instance of enum("red", "green", "blue") returns false, while "red" cast as enum("red") instance of enum("red", "green", "blue") returns true. However, the coercion rules ensure that where a variable or function declaration specifies an enumeration type as the required type, a string (or indeed an xs:untypedAtomic or xs:anyURI value) equal to one of the enumerated values will be accepted.

declare type my:color := enum("red", "green", "orange");
declare type my:fruit := enum("apple", "orange", "banana");
declare variable $orange-color as my:color := "orange";
declare variable $orange-fruit as my:fruit := "orange";

<xsl:item-type name="my:color" as="enum('red', 'green', 'orange')"/>
<xsl:item-type name="my:fruit" as="enum('apple', 'orange', 'banana')"/>
<xsl:variable name="orange-color" as="my:color" select="'orange'"/>
<xsl:variable name="orange-fruit" as="my:fruit" select="'orange'"/>

3.2.7 Node Types

Node types are item types whose instances are all nodes.

The syntax for node types is also used for node tests within path expressions. This explains why the production rules have names such as NodeTest rather than NodeType.

Some of the constructs described in this section include a TypeName. This appears as T in:

• element(N, T)

• attribute(N, T)

• document-node(element(N, T))

The type name T is expanded using the default type namespace rule. The resulting expanded QName must identify a type in the in-scope schema definitions. This can be any schema type: either a simple type, or (except in the case of attributes) a complex type. If it is a simple type then it can be an atomic, union, or list type. It can be a built-in type (such as xs:integer) or a user-defined type. It must however be the name of a type defined in a schema; it cannot be a named item type.

3.2.8 Function, Map, and Array Types

The following sections describe the syntax for item types for functions, including arrays and maps.

The subtype relation among these types is described in the various subsections of 3.3.2 Subtypes of Item Types.

declare record my:list (value as item()*, next? as my:list);

<xsl:record-type name="my:list">
   <xsl:field name="value" as="item()*"/>
   <xsl:field name="next" as="my:list" required="no"/>
</xsl:record-type>

{ "value": 1, "next": { "value": 2, "next": { "value": 3 } } }

declare record t:binary-tree 
   ( left? as t:binary-tree, 
     value as item()*, 
     right? as t:binary-tree
   )

declare function t:values($tree as t:binary-tree?) as item()* {
  $tree ! (t:values(?left), ?value, t:values(?right))   
}

declare record t:tree as (value, children as t:tree*);

declare function t:flatten($tree as t:tree) as item()* {
  $tree?value, $tree?children ! t:flatten(.))   
}

declare record ElementDeclaration (
   name as xs:NCName,
   targetNamespace? as xs:anyURI,
   typeDefinition as (SimpleTypeDefinition | ComplexTypeDefinition),
   nillable as xs:boolean,
   abstract as xs:boolean
);
declare record SimpleTypeDefinition (
   name? as xs:NCName,
   targetNamespace? as xs:anyURI,
   baseType? as SimpleTypeDefinition,
   variety as enum("atomic", "list", "union"),
   facets as Facet*,
);
declare record ComplexTypeDefinition (
   name? as xs:NCName,
   targetNamespace? as xs:anyURI,
   baseType? as ComplexTypeDefinition,
   derivationMethod as enum("extension", "restriction"),
   contentType as record (
      variety as enum("empty", "simple", "element-only", "mixed"),
      particle? as Particle,
      simpleTypeDefinition? as SimpleTypeDefinition
   )
);
declare record Particle (
   minOccurs as xs:nonNegativeInteger,
   maxOccurs as (xs:positiveInteger | enum("unbounded")),
   term as (ElementDeclaration | Wildcard | Group)
);

3.2.9 The type xs:error

The item type xs:error has an empty value space; it never appears as a dynamic type or as the content type of a dynamic element or attribute type. It was defined in XML Schema in the interests of making the type system complete and closed, and it is also available in XPath 4.0 for similar reasons.

3.3.1 Subtypes of Sequence Types

We use the notation A ⊑ B, or subtype(A, B) to indicate that a sequence type A is a subtype of a sequence type B. This section defines the rules for deciding whether any two sequence types have this relationship.

To define the rules, we divide sequence types into six categories:

• The category empty includes the sequence types empty-sequence(), xs:error* and xs:error?. All these sequence types match the empty sequence as their only instance.

• The category void includes the sequence types xs:error and xs:error+, which have no instances (not even the empty sequence).

• The categories X?, X*, X and X+ includes all sequence types having an item type X other than xs:error, together with an occurrence indicator of ? (zero or more), * (one or more), absent (exactly one), or + (one or more) respectively. We use the notation X_i to indicate the item type of such a sequence type.

The judgement A ⊑ B is then determined by the categories of the two sequence types, as defined in the table below. In many cases this depends on the relationship between the item types of A and B. This is denoted using the notation Ai ⊆ Bi , as defined in 3.3.2 Subtypes of Item Types.

3.3.2 Subtypes of Item Types

We use the notation A ⊆ B, or itemtype-subtype(A, B) to indicate that an item type A is a subtype of an item type B. This section defines the rules for deciding whether any two item types have this relationship.

The rules in this section apply to item types, not to item type designators. For example, if the name STR has been defined in the static context as a named item type referring to the type xs:string, then anything said here about the type xs:string applies equally whether it is designated as xs:string or as STR, or indeed as the parenthesized forms (xs:string) or (STR).

References to named item types are handled as described in 3.3.2.9 Subtyping of Named Item Types.

The relationship A ⊆ B is true if and only if at least one of the conditions listed in the following subsections applies:

3.4.2 Item Coercion Rules

The rules in this section are used to process each item J in a supplied sequence, given a required item type R.

1. If R is a generalized atomic type (for example, if it is an atomic type, a pure union type, or an enumeration type), and J is not an atomic item, then:

   1. J is atomized to produce a sequence of atomic items JJ.

   2. Each atomic item in JJ is coerced to the required type R by recursive application of the item coercion rules (the rules in this section) to produce a value V.

   3. The result is the sequence-concatenation of the V values.

   Note:

   For example, if J is an element with type annotation xs:integer, and R is the union type xs:numeric, then the effect is to atomize the element to an xs:integer, and then to coerce the resulting xs:integer to xs:numeric (which leaves the integer unchanged). This is not the same as attempting to coerce the element to each of the alternatives of the union type in turn, which would deliver an instance of xs:double.

2. Otherwise, if R is a choice item type or a pure union type (which includes the case where it is an enumeration type), then:

   1. If J matches (is an instance of) one of the alternatives in R, then J is coerced to the first alternative in R that J matches.

      Note:

      There are two situations where coercing an item to a type that it already matches does not simply return the item unchanged:

      • When the required type is a typed function type (see 3.2.8.1 Function Types), then function coercion is applied to coerce J to that function type, as described in 3.4.4 Function Coercion.

      • When the required type is a record type and the supplied value is a map, then coercion may change the entry order^DM of the entries in the map.

   2. Otherwise, the item coercion rules (the rules in this section) are applied to J recursively with R set to each of the alternatives in the choice or union item type, in order, until an alternative is found that does not result in a type error; a type error is raised only if all alternatives fail.

      The error code used in the event of failure should be the error code arising from the first unsuccessful matching attempt. (The diagnostic information associated with the error may also describe how further attempts failed.)

   Note:

   Suppose the required type is (xs:integer | element(e))* and the supplied value is the sequence (<e>22</e>, 23, <f>24</f>). Item coercion is applied independently to each of the three items in this sequence. The first item matches one of the alternatives, namely element(e), so it is returned unchanged as an element node. The second item (the integer 23) also matches one of the alternatives, and is returned unchanged as an integer. The third item does not match any of the alternatives, so coercion is attempted to each one in turn. Coercion to type xs:integer succeeds (by virtue of atomization and untyped atomic conversion), so the final result is the sequence (<e>22</e>, 23, 24)

   Note:

   Suppose the required type is enum("red", "green", "blue") and the supplied value is "green". The enumeration type is defined as a choice item type whose alternatives are singleton enumerations, so the rules are applied first to the type enum("red") (which fails), and then to the type enum("green") (which succeeds). The strings in an enumeration type are required to be distinct so the order of checking is in this case immaterial. The supplied value will be accepted, and will be relabeled as an instance of enum("green"), which is treated as a schema type equivalent to a type derived from xs:string by restriction.

   Note:

   Schema-defined union types behave in exactly the same way as choice item types.

3. If R is an atomic type and J is an atomic item, then:

   1. If J is an instance of R then it is used unchanged.

   2. If J is an instance of type xs:untypedAtomic then:

      1. If R is namespace-sensitive then a type error [err:XPTY0117] is raised.

      2. Otherwise, J is cast to type R.

4. If there is an entry (from, to) in the following table such that J is an instance of from, and to is R, then J is cast to type R.

   Implicit Casting

   from	to
   xs:decimal	xs:double
   xs:double	xs:decimal
   xs:decimal	xs:float
   xs:float	xs:decimal
   xs:float	xs:double
   xs:double	xs:float
   xs:string	xs:anyURI
   xs:anyURI	xs:string
   xs:hexBinary	xs:base64Binary
   xs:base64Binary	xs:hexBinary

   Note:

   The item type in the to column must match R exactly; however, J may belong to a subtype of the type in the from column.

   For example, an xs:NCName will be cast to type xs:anyURI, but an xs:anyURI will not be cast to type xs:NCName.

   Similarly, an xs:integer will be cast to type xs:double, but an xs:double will not be cast to type xs:integer.

5. If R is derived from some primitive atomic type P, then J is relabeled as an instance of R if it satisfies all the following conditions:

   • J is an instance of P.

   • J is not an instance of R.

   • The datum^DM of J is within the value space of R.

   Relabeling an atomic item changes the type annotation but not the datum^DM. For example, the xs:integer value 3 can be relabeled as an instance of xs:unsignedByte, because the datum is within the value space of xs:unsignedByte.

   Note:

   Relabeling is not the same as casting. For example, the xs:decimal value 10.1 can be cast to xs:integer, but it cannot be relabeled as xs:integer, because its datum not within the value space of xs:integer.

   Note:

   The effect of this rule is that if, for example, a function parameter is declared with an expected type of xs:positiveInteger, then a call that supplies the literal value 3 will succeed, whereas a call that supplies -3 will fail.

   This differs from previous versions of this specification, where both these calls would fail.

   This change allows the arguments of existing functions to be defined with a more precise type. For example, the $position argument of array:get could be defined as xs:positiveInteger rather than xs:integer.

   Note:

   If T is a union type with members xs:negativeInteger and xs:positiveInteger and the supplied value is the sequence (20, -20), then the effect of these rules is that the first item 20 is relabeled as type xs:positiveInteger and the second item -20is relabeled as type xs:negativeInteger.

   Note:

   Promotion (for example of xs:float to xs:double) occurs only when T is a primitive type. Relabeling occurs only when T is a derived type. Promotion and relabeling are therefore never combined.

   Note:

   A singleton enumeration type such as enum("green") is treated as an atomic type derived by restriction from xs:string; so if the xs:string value "green" is supplied in a context where the required type is enum("red", "green", "blue"), the value will be accepted and will be relabeled as an instance of enum("green").

6. If R is an ArrayType other than array(*) and J is an array, then J is converted to a new array by converting each member to the required member type by applying the coercion rules recursively.

   Note:

   For example, if the required type is array(xs:double) and the supplied value is [ 1, 2 ], the array is converted to [ 1e0, 2e0 ].

7. If R is a MapType other than map(*) and J is a map, then J is converted to a new map as follows:

   1. Each key in the supplied map is converted to the required map key type by applying the coercion rules. If the resulting map would contain duplicate keys, a type error is raised [err:XPTY0004].

   2. The corresponding value is converted to the required map value type by applying the coercion rules recursively.

   3. The order of entries in the map remains unchanged.

   Note:

   For example, if the required type is map(xs:string, xs:double) and the supplied value is { "x": 1, "y": 2 }, the map is converted to { "x": 1e0, "y": 2e0 }.

   Note:

   Duplicate keys can occur if the value space of the target type is more restrictive than the original type. For example, an error is raised if the map { 1.2: 0, 1.2000001: 0 }, which contains two keys of type xs:decimal, is coerced to the type map(xs:float, xs:integer).

8. If R is a RecordType and J is a map, then J is converted to a new map as follows:

   1. The keys in the supplied map are unchanged.

   2. In any map entry whose key is equal to the name of one of the field declarations in R (under the rules of the atomic-equal function), the corresponding value is converted to the required type defined by that field declaration, by applying the coercion rules recursively (but with XPath 1.0 compatibility mode treated as false).

   3. The order of entries in the map is changed: entries whose keys correspond to the names of field declarations in R appear first, in the order of the corresponding field declarations, and (if the record type is extensible) other entries then follow retaining their relative order in J.

   Note:

   For example, if the required type is record(longitude as xs:double, latitude as xs:double) and the supplied value is { "latitude": 53.2, "longitude": 0 }, then the map is converted to { "longitude": 0.0e0, "latitude": 53.2e0 }.

9. If R is a TypedFunctionType and J is a function item, then function coercion is applied to J.

   Note:

   Function coercion applies even if J is already an instance of R.

   Maps and arrays are functions, so function coercion applies to them as well.

10. If, after the above conversions, the resulting item does not match the expected item type R according to the rules for SequenceType Matching, a type error is raised [err:XPTY0004].

    Note:

    Under the general rules for type errors (see 2.4.1 Kinds of Errors), a processor may report a type error during static analysis if it will necessarily occur when the expression is evaluated. For example, the function call fn:abs("beer") will necessarily fail when evaluated, because the function requires a numeric value as its argument; this may be detected and reported as a static error.