This document is also available in these non-normative formats: Specification in XML format and XML function catalog.
Copyright © 2000 W3C® (MIT, ERCIM, Keio, Beihang). W3C liability, trademark and document use rules apply.
This document defines constructor functions, operators, and functions on the datatypes defined in [XML Schema Part 2: Datatypes Second Edition] and the datatypes defined in [XQuery and XPath Data Model (XDM) 4.0]. It also defines functions and operators on nodes and node sequences as defined in the [XQuery and XPath Data Model (XDM) 4.0]. These functions and operators are defined for use in [XML Path Language (XPath) 4.0] and [XQuery 4.0: An XML Query Language] and [XSL Transformations (XSLT) Version 4.0] and other related XML standards. The signatures and summaries of functions defined in this document are available at: http://www.w3.org/2005/xpath-functions/.
A summary of changes since version 3.1 is provided at H Changes since 3.1.
This section describes the status of this document at the time of its publication. Other documents may supersede this document.
This document is a working draft developed and maintained by a W3C Community Group, the XQuery and XSLT Extensions Community Group unofficially known as QT4CG (where "QT" denotes Query and Transformation). This draft is work in progress and should not be considered either stable or complete. Standard W3C copyright and patent conditions apply.
The community group welcomes comments on the specification. Comments are best submitted as issues on the group's GitHub repository.
The community group maintains two extensive test suites, one oriented to XQuery and XPath, the other to XSLT. These can be found at qt4tests and xslt40-test respectively. New tests, or suggestions for correcting existing tests, are welcome. The test suites include extensive metadata describing the conditions for applicability of each test case as well as the expected results. They do not include any test drivers for executing the tests: each implementation is expected to provide its own test driver.
The publications of this community group are dedicated to our co-chair, Michael Sperberg-McQueen (1954–2024).
These functions in this section access resources external to a query or stylesheet, and convert between external file formats and their XPath and XQuery data model representation.
The functions listed in this section parse or serialize JSON data.
JSON is a popular format for exchange of structured data on the web: it is specified in [RFC 7159]. This section describes facilities allowing JSON data to be converted to and from XDM values.
This specification describes two ways of representing JSON data losslessly using XDM constructs. The first method uses XDM maps to represent JSON objects, and XDM arrays to represent JSON arrays. The second method represents all JSON constructs using XDM element and attribute nodes.
| Function | Meaning |
|---|---|
fn:parse-json | Parses input supplied in the form of a JSON text, returning the results typically in the form of a map or array. |
fn:json-doc | Reads an external resource containing JSON, and returns the result of parsing the resource as JSON. |
fn:json-to-xml | Parses a string supplied in the form of a JSON text, returning the results in the form of an XML document node. |
fn:xml-to-json | Converts an XML tree, whose format corresponds to the XML representation of JSON defined in this specification, into a string conforming to the JSON grammar. |
Note also:
The function fn:serialize has an option to generate JSON output from a structure of maps and arrays.
The function fn:element-to-map enables arbitrary XML node trees to be converted to trees of maps and arrays suitable for serializing as JSON.
The rules regarding use of non-XML characters in JSON texts have been relaxed. [Issue 414 PR 546 25 July 2023]
An option is provided to control how the JSON null value should be handled. [Issue 960 PR 1028 20 February 2024]
An option is provided to control how JSON numbers should be formatted. [Issues 973 1037 PRs 975 1058 1246 12 March 2024]
The default for the escape option has been changed to false. The 3.1 specification gave the default value as true, but this appears to have been an error, since it was inconsistent with examples given in the specification and with tests in the test suite. [Issue 1555 PR 1565 11 November 2024]
The order of entries in maps is retained. [Issue 1651 PR 1703 14 January 2025]
Support for binary input has been added. [Issue 748 PR 2013 20 May 2025]
Parses input supplied in the form of a JSON text, returning the results typically in the form of a map or array.
fn:parse-json( | ||
$value | as , | |
$options | as | := {} |
) as | ||
This function is deterministic, context-independent, and focus-independent.
If the second argument is omitted or an empty sequence, the result is the same as calling the two-argument form with an empty map as the value of the $options argument.
The first argument is a JSON text as defined in [RFC 7159], in the form of a string or binary value. The function parses this input to return an XDM value.
If $value is the empty sequence, the function returns the empty sequence.
Note:
If the input is "null", the result will also be an empty sequence.
If $value is supplied as a binary value, it is converted to a string. The function detects the encoding using the same rules as the unparsed-text function, except that the special handling of media types such as text/xml and application/xml may be skipped. Otherwise, if $value is a string, it is processed unchanged.
The $options argument can be used to control the way in which the parsing takes place. The option parameter conventions apply.
The entries that may appear in the $options map are as follows:
record( | |
liberal? | as xs:boolean, |
duplicates? | as xs:string, |
escape? | as xs:boolean, |
fallback? | as (fn(xs:string) as xs:anyAtomicType)?, |
null? | as item()*, |
number-parser? | as (fn(xs:untypedAtomic) as item()?)? |
) | |
| Key | Value | Meaning |
|---|---|---|
| Determines whether deviations from the syntax of RFC7159 are permitted.
| |
false | The input must consist of an optional byte order mark (which is ignored) followed by a string that conforms to the grammar of JSON-text in [RFC 7159]. An error must be raised [err:FOJS0001] if the input does not conform to the grammar. | |
true | The input may contain deviations from the grammar of [RFC 7159], which are handled in an implementation-defined way. (Note: some popular extensions include allowing quotes on keys to be omitted, allowing a comma to appear after the last item in an array, allowing leading zeroes in numbers, and allowing control characters such as tab and newline to be present in unescaped form.) Since the extensions accepted are implementation-defined, an error may be raised [err:FOJS0001] if the input does not conform to the grammar. | |
| Determines the policy for handling duplicate keys in a JSON object. To determine whether keys are duplicates, they are compared using the Unicode codepoint collation, after expanding escape sequences, unless the escape option is set to true, in which case keys are compared in escaped form.
| |
reject | An error is raised [err:FOJS0003] if duplicate keys are encountered. | |
use-first | If duplicate keys are present in a JSON object, all but the first of a set of duplicates are ignored. | |
use-last | If duplicate keys are present in a JSON object, all but the last of a set of duplicates are ignored. | |
| Determines whether special characters are represented in the XDM output in backslash-escaped form.
| |
false | Any permitted character in the input, whether or not it is represented in the input by means of an escape sequence, is represented as an unescaped character in the result. Any other character or codepoint (for example, an unpaired surrogate) is passed to the fallback function as described below; in the absence of a fallback function, it is replaced by U+FFFD (REPLACEMENT CHARACTER, �) . | |
true | JSON escape sequences are used in the result to represent special characters in the JSON input, as defined below, whether or not they were represented using JSON escape sequences in the input. The characters that are considered “special” for this purpose are:
\t), or a six-character escape sequence otherwise (for example \uDEAD). Characters other than these are not escaped in the result, even if they were escaped in the input. | |
| Provides a function which is called when the input contains an escape sequence that represents a character that is not a permitted character. It is an error to supply the fallback option if the escape option is present with the value true.
| |
User-supplied function | The function is called when the JSON input contains character that is not a permitted character It is called once for any surrogate that is not properly paired with another surrogate. The untyped atomic item supplied as the argument will always be a two- or six-character escape sequence, starting with a backslash, that conforms to the rules in the JSON grammar (as extended by the implementation if liberal:true() is specified): for example \b or \uFFFF or \uDEAD. By default, the escape sequence is replaced with the Unicode | |
| Determines how the JSON null value should be represented.
| |
Value | The supplied XDM value is used to represent the JSON null value. The default representation of null is an empty sequence, which works well in cases where setting a property of an object to null has the same meaning as omitting the property. It works less well in cases where null is used with some other meaning, because expressions such as the lookup operator ? flatten the result to a single sequence of items, which means that any entries whose value is an empty sequence effectively disappear. The property can be set to any XDM value; a suggested value is the xs:QName value fn:QName("http://www.w3.org/2005/xpath-functions", "null"), which is recognized by the JSON serialization method as representing the JSON value null. | |
| Determines how numeric values should be processed.
| |
User-supplied function | The supplied function is called to process the string value of any JSON number in the input. By default, numbers are processed by converting to xs:double using the XPath casting rules. Supplying the value xs:decimal#1 will instead convert to xs:decimal (which potentially retains more precision, but disallows exponential notation), while supplying a function that casts to (xs:decimal | xs:double) will treat the value as xs:decimal if there is no exponent, or as xs:double otherwise. Supplying the value fn:identity#1 causes the value to be retained unchanged as an xs:untypedAtomic. If the liberal option is false (the default), then the supplied number-parser is called if and only if the value conforms to the JSON grammar for numbers (for example, a leading plus sign and redundant leading zeroes are not allowed). If the liberal option is true then it is also called if the value conforms to an implementation-defined extension of this grammar. | |
The various structures that can occur in JSON are transformed recursively to XDM values as follows:
A JSON object is converted to a map. The entries in the map correspond to the key/value pairs in the JSON object. The key is always of type xs:string; the associated value may be of any type, and is the result of converting the JSON value by recursive application of these rules. For example, the JSON text { "x": 2, "y": 5 } is transformed to the value { "x": 2, "y": 5 }.
If duplicate keys are encountered in a JSON object, they are handled as determined by the duplicates option defined above.
The order of entries is retained.
A JSON array is transformed to an array whose members are the result of converting the corresponding member of the array by recursive application of these rules. For example, the JSON text [ "a", "b", null ] is transformed (by default) to the value [ "a", "b", () ].
A JSON string is converted to an xs:string value. The handling of special characters depends on the escape and fallback options, as described in the table above.
A JSON number is processed using the function supplied in the number-parser option; by default it is converted to an xs:double value using the rules for casting from xs:string to xs:double.
The JSON boolean values true and false are converted to the corresponding xs:boolean values.
The JSON value null is converted to the value given by the null option, which defaults to an empty sequence.
A dynamic error [err:FOJS0001] occurs if the value of $value does not conform to the JSON grammar, unless the option "liberal":true() is present and the processor chooses to accept the deviation.
A dynamic error [err:FOJS0003] occurs if the option "duplicates": "reject" is present and the value of $value contains a JSON object with duplicate keys.
A dynamic error [err:FOJS0005] occurs if the $options map contains an entry whose key is defined in this specification and whose value is not valid for that key, or if it contains an entry with the key fallback when the option "escape":true() is also present.
The result of the function will be an instance of one of the following types. An instance of test (or in XQuery, typeswitch) can be used to distinguish them:
map(xs:string, item()?) for a JSON object
array(item()?) for a JSON array
xs:string for a JSON string
xs:double for a JSON number
xs:boolean for a JSON boolean
empty-sequence() for a JSON null (or for empty input)
If the input starts with a byte order mark, this function ignores it. The byte order mark may have been added to the data stream in order to facilitate decoding of an octet stream to a character string, but since this function takes a character string as input, the byte order mark serves no useful purpose.
The possibility of the input containing characters that are not valid in XML (for example, unpaired surrogates) arises only when such characters are expressed using JSON escape sequences. This is because the input to the function is an instance of xs:string, which by definition (see [XQuery and XPath Data Model (XDM) 4.0] section 4.1.5 XML and XSD Versions) cannot contain unpaired surrogates.
The serializer provides an option to output data in json-lines format. This is a format for structured data containing one JSON value (usually but not necessarily a JSON object) on each line. There is no corresponding option to parse json-lines input, but this can be achieved using the expression unparsed-text-lines($uri) =!> parse-json().
| Expression: |
|
|---|---|
| Result: |
|
| Expression: |
|
| Result: |
|
| Expression: |
|
| Result: |
|
| Expression: | parse-json(
'{ "x": "\\", "y": "\u0025" }',
{ 'escape': true() }
) |
| Result: | { "x": "\\", "y": "%" } |
| Expression: | parse-json(
'{ "x": "\\", "y": "\u0000" }'
) |
| Result: | { "x": "\", "y": char(0xFFFD) } |
| Expression: | parse-json(
'{ "x": "\\", "y": "\u0000" }',
{ 'escape': true() }
) |
| Result: | { "x": "\\", "y": "\u0000" } |
| Expression: | parse-json(
'{ "x": "\\", "y": "\u0000" }',
{ 'fallback': fn($s) { '[' || $s || ']' } }
) |
| Result: | { "x": "\", "y": "[\u0000]" } |
| Expression: | parse-json(
"1984.2",
{ 'number-parser': fn { xs:integer(round(.)) } }
) |
| Result: | 1984 |
| Expression: | parse-json(
'[ 1, -1, 2 ]',
{ 'number-parser': fn { boolean(. >= 0) } }
) |
| Result: | [ true(), false(), true() ] |
| Expression: | parse-json('[ "a", null, "b" ]',
{ 'null': #fn:null }
) |
| Result: | [ "a", #fn:null, "b" ] |
Additional options are available, as defined by fn:parse-json.
It is no longer automatically an error if the input contains a codepoint that is not valid in XML. Instead, the codepoint must be a permitted character. The set of permitted characters is implementation-defined, but it is recommended that all Unicode characters should be accepted. [Issue 414 PR 546 25 July 2023]
Reads an external resource containing JSON, and returns the result of parsing the resource as JSON.
fn:json-doc( | ||
$source | as , | |
$options | as | := {} |
) as | ||
This function is deterministic, context-dependent, and focus-independent. It depends on executable base URI.
If the second argument is omitted or an empty sequence, the result is the same as calling the two-argument form with an empty map as the value of the $options argument.
The effect of the two-argument function call fn:json-doc($H, $M)is equivalent to the function composition fn:unparsed-binarytext($H) => fn:parse-json($M)., except that:
The function may accept a resource in any encoding. [RFC 7159] requires UTF-8, UTF-16, or UTF-32 to be accepted, but it is not an error if a different encoding is used. Unless external encoding information is available, the function must assume that the encoding is one of UTF-8, UTF-16, or UTF-32, and must distinguish these cases by examination of the initial octets of the resource.
Having established the encoding, the function must accept any codepoint that can validly occur in a JSON text, with the exception of unpaired surrogates.
If $source is the empty sequence, the function returns the empty sequence.
The function may raise any error defined for the fn:unparsed-text or fn:parse-json functions.
An initial byte order mark is dropped, as with the fn:unparsed-text function.
If the input cannot be decoded (that is, converted into a sequence of Unicode codepoints, which may or may not represent characters), then a dynamic error occurs as with the fn:unparsed-text function.
If the input can be decoded, then the possibility still arises that the resulting sequence of codepoints includes codepoints that are not permitted characters. Such codepoints are translated into JSON escape sequences (for example, \uFFFF), and the JSON escape sequence is then passed to the fallback function specified in the $options argument, which in turn defaults to a function that returns the Unicode REPLACEMENT CHARACTER (xFFFD).
The function may accept a resource in any encoding. [RFC 7159] requires UTF-8, UTF-16, or UTF-32 to be accepted, but it is not an error if a different encoding is used. The function detects the encoding using the same rules as the unparsed-text function, except that the special handling of media types such as text/xml and application/xml may be skipped.
This section describes functions that parse CSV data.
[Definition] The term comma separated values or CSV refers to a wide variety of plain-text tabular data formats with fields and records separated by standard character delimiters (often, but not invariably, commas).
A CSV is a 2-dimensional tabular data structure consisting of multiple rows (also known as records). Each row contains multiple fields. Fields occupying the same position in successive rows constitute a column. Columns are identified by position and optionally by name. Column names can be assigned within a CSV using an optional header row.
CSV has developed informally for decades, and many variations are found. This specification refers to [RFC 4180], which provides a standardized grammar. This specification extends the grammar defined in [RFC 4180] as follows:
This specification uses the term row where RFC 4180 uses record.
Line endings are normalized: specifically, the character sequences U+000D (CARRIAGE RETURN) , or U+000D (CARRIAGE RETURN) followed by U+000A (NEWLINE) , are converted to a single U+000A (NEWLINE) character. This applies whether or not the line ending appears within a quoted string, and whether or not U+000A (NEWLINE) is the chosen row delimiter.
Row delimiters other than newline are recognized.
Field delimiters other than U+002C (COMMA, ,) are recognized.
Quote characters other than U+0022 (QUOTATION MARK, ") are recognized.
Non-ASCII characters are recognized.
This specification defines a mapping from this extended grammar to constructs in the XDM model, and provides illustrative examples of how these constructs can be combined with other language features to process CSV data.
| Function | Meaning |
|---|---|
fn:csv-to-arrays | Parses CSV data supplied as a string, returning the results in the form of a sequence of arrays of strings. |
fn:parse-csv | Parses CSV data, returning the results in the form of a record containing information about the names in the header, as well as the data itself. |
fn:csv-doc | Reads an external resource containing CSV, and returns the results as a record containing information about the names in the header, as well as the data itself. |
fn:csv-to-xml | Parses CSV data supplied as a string, returning the results as an XML document, as described by 17.5.9 Representing CSV data as XML. |
The most basic function for parsing CSV is fn:csv-to-arrays which recognizes the delimiters for rows and fields and returns a sequence of arrays each corresponding to one row. The fields within each array are represented as instances of xs:string.
The other two functions recognize column names, and make it easier to address individual fields using these names. The parse-csv function delivers this capability using XDM maps and functions, while csv-to-xml function represents the information using XDM element nodes.
Parses CSV data, returning the results in the form of a record containing information about the names in the header, as well as the data itself.
fn:parse-csv( | ||
$value | as , | |
$options | as | := {} |
) as parsed-csv-structure-record? | ||
This function is deterministic, context-independent, and focus-independent.
If $value is the empty sequence, the function returns the empty sequence.
If $value is supplied as a binary value, it is converted to a string. The function detects the encoding using the same rules as the unparsed-text function, except that the special handling of media types such as text/xml and application/xml may be skipped. Otherwise, if $value is a string, it is processed unchanged.
The resulting input supplied in $value is CSV data, as defined in [RFC 4180]. The function first parses the input using fn:csv-to-arrays, and then further processes the result. The initial parsing is exactly as defined for fn:csv-to-arrays, and can be controlled using the same options. Additional options are available to control the way in which header information and column names are handled.
If the input is the a zero-length string, the function returns a parsed-csv-structure-record whose rows entry is the empty sequence.
The $options argument can be used to control the way in which the parsing takes place. The option parameter conventions apply.
If the $options argument is omitted or is an empty sequence, the result is the same as calling the two-argument form with an empty map as the value of the $options argument.
The entries that may appear in the $options map are as follows:
record( | |
field-delimiter? | as xs:string, |
row-delimiter? | as xs:string, |
quote-character? | as xs:string, |
trim-whitespace? | as xs:boolean, |
header? | as item()*, |
select-columns? | as xs:positiveInteger*, |
trim-rows? | as xs:boolean |
) | |
| Key | Value | Meaning |
|---|---|---|
| The character used to delimit fields within a record. An instance of xs:string whose length is exactly one.
| |
| The character used to delimit rows within the CSV string. An instance of xs:string whose length is exactly one. Defaults to a single newline character (U+000A (NEWLINE) ). Note that this is tested after line endings are normalized.
| |
| The character used to quote fields within the CSV string. An instance of xs:string whose length is exactly one.
| |
| Determines whether leading and trailing whitespace is removed from the content of unquoted fields.
| |
false | Unquoted fields will be returned with any leading or trailing whitespace intact. | |
true | Unquoted fields will be returned with leading or trailing whitespace removed, and all other whitespace preserved. | |
| Determines whether the first row of the CSV should be treated as a list of column names, or whether column names are being supplied by the caller. The value must either be a single boolean, or a sequence of zero or more strings.
| |
true | Column names are taken from the first row of the CSV data. | |
false | Column names are not available; all references to columns are by ordinal position. | |
xs:string* | Supplies explicit names for the columns. The Nth name in the list applies to the Nth column after any filtering or rearrangement. A zero-length string can be used when there is a column that requires no name. | |
| A sequence of integers indicating which columns to include and in which order. If this option is absent or empty, all columns are returned in their original order. For example, the value 1 to 4 indicates that the output contains the first, second, third, and fourth columns from the input, in order, while (1, 5, 4) indicates that the output contains three columns, taken from the first, fifth, and fourth columns of the input, in that order. An integer in the sequence is treated as the 1-based index of the column to include. Any other columns are dropped. If a particular row includes no field at the specified index, an empty field is included at the relevant position in the result. If an integer appears more than once then the result will include duplicated columns.
| |
| Determines whether all rows should be adjusted to contain the same number of fields. This option is ignored if select-columns is specified.
| |
false | No padding or trimming of rows takes place, unless requested using the select-columns option. | |
true | The number of fields in the first row (whether this be a header or a data row) determines the number of fields in every subsequent row; to achieve this, excess fields are removed, or additional zero-length fields are added. | |
The result of the function is a parsed-csv-structure-record, as defined in 17.5.6 Record fn:parsed-csv-structure-record.
A dynamic error [err:FOCV0001] occurs if the value of $csv does not conform to the required grammar.
A dynamic error [err:FOCV0002] occurs if any of the options field-delimiter, row-delimiter, or quote-character is not a single character.
A dynamic error [err:FOCV0003] occurs if the same character is used for more than one of the options field-delimiter, row-delimiter, and quote-character.
The default row delimiter is a single newline character U+000A (NEWLINE) . Alternative line endings such as CR and CRLF will already have been normalized to a single newline.
All fields are returned as xs:string values.
Quoted fields in the input are returned without the quotes.
For more discussion of the returned data, see 17.5.5 Enhanced parsing of CSV data to maps and arrays.
| Variables | |
|---|---|
let $display := fn($result) {
(: tidy up the result for display (function items cannot be properly displayed) :)
map:put($result, "get", "(: function :)")
} | |
Default delimiters, no column headers: | |
| Expression: | let $input := string-join(
("name,city", "Bob,Berlin", "Alice,Aachen"),
char('\n')
)
let $result := parse-csv($input)
return (
$result => $display(),
$result?get(1, 2),
$result?get(2, 2)
) |
|---|---|
| Result: | {
"columns": (),
"column-index": {},
"rows": ([ "name", "city" ], [ "Bob", "Berlin" ], [ "Alice", "Aachen" ]),
"get": "(: function :)"
},
"city",
"Berlin" |
Default delimiters, column headers: | |
| Expression: | let $input := string-join(
("name,city", "Bob,Berlin", "Alice,Aachen"),
char('\n')
)
let $result := parse-csv($input, { "header": true() })
return (
$result => $display(),
$result?get(1, "name"),
$result?get(2, "city")
) |
| Result: | {
"columns": ("name", "city"),
"column-index": { "name": 1, "city": 2 },
"rows": ([ "Bob", "Berlin" ], [ "Alice", "Aachen" ]),
"get": "(: function :)"
},
"Bob",
"Aachen" |
Custom delimiters, no column headers: | |
| Expression: | let $options := {
"row-delimiter": "§",
"field-delimiter": ";",
"quote-character": "|"
}
let $input := "|name|;|city|§|Bob|;|Berlin|§|Alice|;|Aachen|"
let $result := parse-csv($input, $options)
return (
$result => $display(),
$result?get(3, 1)
) |
| Result: | {
"columns": (),
"column-index": {},
"rows": ([ "name", "city" ], [ "Bob", "Berlin" ], [ "Alice", "Aachen" ]),
"get": "(: function :)"
},
"Alice" |
Supplied column names: | |
| Expression: | let $headers := ("Person", "Location")
let $options := { "header": $headers, "row-delimiter": ";" }
let $input := "Alice,Aachen;Bob,Berlin;"
let $parsed-csv := parse-csv($input, $options)
return (
$parsed-csv => $display(),
$parsed-csv?get(2, "Location")
) |
| Result: | {
"columns": ("Person", "Location"),
"column-index": { "Person": 1, "Location": 2 },
"rows": ([ "Alice", "Aachen" ], [ "Bob", "Berlin" ]),
"get": "(: function :)"
},
"Berlin" |
Filtering columns, with ragged input and | |
| Expression: | let $input := string-join((
"date,name,city,amount,currency,original amount,note",
"2023-07-19,Bob,Berlin,10.00,USD,13.99",
"2023-07-20,Alice,Aachen,15.00",
"2023-07-20,Charlie,Celle,15.00,GBP,11.99,cake,not a lie"
), char('\n'))
let $options := {
"header": true(),
"select-columns": (2, 1, 4)
}
let $result := parse-csv($input, $options)
return (
$result => $display(),
$result?get(2, "amount")
) |
| Result: | {
"columns": ("name", "date", "amount"),
"column-index": { "name": 1, "date": 2, "amount": 3 },
"rows": (
[ "Bob", "2023-07-19", "10.00" ],
[ "Alice", "2023-07-20", "15.00" ],
[ "Charlie", "2023-07-20", "15.00" ]
),
"get": "(: function :)"
},
"15.00" |
Filtering columns, with supplied column map | |
| Expression: | let $input := string-join((
"2023-07-20,Alice,Aachen,15.00",
"2023-07-19,Bob,Berlin,10.00,USD,13.99",
"2023-07-20,Charlie,Celle,15.00,GBP,11.99,cake,not a lie"
), char('\n'))
let $options := {
"header": ( "Person", "", "Amount" ),
"select-columns": (2, 1, 4)
}
let $result := parse-csv($input, $options)
return (
$result => $display(),
$result?get(2, "Person"),
$result?get(2, "Amount")
) |
| Result: | {
"columns": ("Person", "", "Amount"),
"column-index": { "Person": 1, "Amount": 3 },
"rows": ([ "Alice", "2023-07-20", "15.00" ],
[ "Bob", "2023-07-19", "10.00" ],
[ "Charlie", "2023-07-20", "15.00" ]),
"get": "(: function :)"
},
"Bob",
"10.00" |
Specifying the number of columns explicitly, with | |
| Expression: | let $input := string-join((
"date, name, amount, currency, original amount",
"2023-07-19,Bob, 10.00, USD, 13.99",
"2023-07-20,Alice, 15.00",
"2023-07-20,Charlie, 15.00, GBP, 11.99, extra data"
), char('\n'))
let $options := {
"header": false(),
"select-columns": 1 to 5,
"trim-whitespace" :true()
}
let $result := parse-csv($input, $options)
return (
$result => $display(),
$result?get(4, 3)
) |
| Result: | {
"columns": (),
"column-index": {},
"rows": (
[ "date", "name", "amount", "currency", "original amount" ],
[ "2023-07-19", "Bob", "10.00", "USD", "13.99" ],
[ "2023-07-20", "Alice", "15.00", "", "" ],
[ "2023-07-20", "Charlie", "15.00", "GBP", "11.99" ]
),
"get": "(: function :)"
},
"15.00" |
Specifying the number of columns with a number and | |
| Expression: | let $input := string-join((
"date,name,city,amount,currency,original amount,note",
"2023-07-19,Bob,Berlin,10.00,USD,13.99",
"2023-07-20,Alice,Aachen,15.00",
"2023-07-20,Charlie,Celle,15.00,GBP,11.99,cake,not a lie"
), char('\n'))
let $options := { "header": true(), "select-columns": 1 to 6 }
let $result := parse-csv($input, $options)
return (
$result => $display(),
$result?get(3, "original amount")
) |
| Result: | {
"columns": ("date", "name", "city",
"amount", "currency", "original amount"),
"column-index": {
"date": 1, "name": 2, "city": 3, "amount": 4,
"currency": 5, "original amount": 6
},
"rows": (
[ "2023-07-19", "Bob", "Berlin", "10.00", "USD", "13.99"],
[ "2023-07-20", "Alice", "Aachen", "15.00", "", ""],
[ "2023-07-20", "Charlie", "Celle", "15.00", "GBP", "11.99"]
),
"get": "(: function :)"
},
"11.99" |
Use the arrows to browse significant changes since the 3.1 version of this specification.
See 1 Introduction
Sections with significant changes are marked Δ in the table of contents. New functions introduced in this version are marked ➕ in the table of contents.
See 1 Introduction
New in 4.0
New in 4.0
See 2.1.12 fn:slice
New in 4.0. The function replaces the internal op:same-key function in 3.1
PR 1120 1150
A callback function can be supplied for comparing individual items.
Changed in 4.0 to use transitive equality comparisons for numeric values.
PR 614 987
New in 4.0
New in 4.0. Originally proposed under the name fn:uniform
New in 4.0. Originally proposed under the name fn:unique
New in 4.0
See 2.5.3 fn:every
New in 4.0
See 2.5.9 fn:highest
New in 4.0
New in 4.0
See 2.5.11 fn:lowest
New in 4.0
New in 4.0
See 2.5.16 fn:some
PR 795 2228
New in 4.0
PR 521 761
New in 4.0
New in 4.0
See 4.4.5 fn:is-NaN
PR 1260 1275
A third argument has been added, providing control over the rounding mode.
See 4.4.6 fn:round
PR 1049 1151
Decimal format parameters can now be supplied directly as a map in the third argument, rather than referencing a format defined in the static context.
PR 1205 1230
New in 4.0
See 4.8.2 math:e
See 4.8.8 math:cosh
See 4.8.15 math:sinh
See 4.8.18 math:tanh
The 3.1 specification suggested that every value in the result range should have the same chance of being chosen. This has been corrected to say that the distribution should be arithmetically uniform (because there are as many xs:double values between 0.01 and 0.1 as there are between 0.1 and 1.0).
PR 261 306 993
New in 4.0
See 5.4.1 fn:char
New in 4.0
PR 937 995 1190
New in 4.0
See 5.4.13 fn:hash
New in 4.0
PR 1423 1413
New in 4.0
New in 4.0
Reformulated in 4.0 in terms of the new fn:in-scope-namespaces function; the semantics are unchanged.
Reformulated in 4.0 in terms of the new fn:in-scope-namespaces function; the semantics are unchanged.
PR 1620 1886
Options are added to customize the form of the output.
See 12.2.9 fn:path
PR 1547 1551
New in 4.0
New in 4.0
New in 4.0
PR 478 515
New in 4.0
PR 1575 1906
A new function fn:element-to-map is provided for converting XDM trees to maps suitable for serialization as JSON. Unlike the fn:xml-to-json function retained from 3.1, this can handle arbitrary XML as input.
New in 4.0
PR 968 1295
New in 4.0
PR 476 1087
New in 4.0
PR 360 476
New in 4.0
New in 4.0
New in 4.0
Supplying an empty sequence as the value of an optional argument is equivalent to omitting the argument.
PR 1117 1279
The $options parameter has been added.
New in 4.0
PR 259 956
A new function is available for processing input data in HTML format.
See 17.3 Functions on HTML Data
New in 4.0
An option is provided to control how JSON numbers should be formatted.
Additional options are available, as defined by fn:parse-json.
New in 4.0
New in 4.0
New in 4.0
PR 629 803
New in 4.0
PR 533 719 834
New functions are available for processing input data in CSV (comma separated values) format.
Comparison of mixed numeric types (for example xs:double and xs:decimal) now generally converts both values to xs:decimal.
PR 289 1901
A third argument is added, allowing user control of how absent keys should be handled.
See 14.4.9 map:get
A third argument is added, allowing user control of how index-out-of-bounds conditions should be handled.
A new collation URI is defined for Unicode case-insensitive comparison and ordering.
PR 1727 1740
It is no longer guaranteed that the new key replaces the existing key.
See 14.4.14 map:put
Generalized to work with JNodes as well as XNodes.
The function is extended to handle JNodes.
See 12.2.9 fn:path
Generalized to work with JNodes as well as XNodes.
Atomic items of types xs:hexBinary and xs:base64Binary are now mutually comparable. In rare cases, where an application uses both types and assumes they are distinct, this can represent a backwards incompatibility.
PR 173
New in 4.0
See 18.4 fn:op
PR 203
New in 4.0
See 14.4.1 map:build
PR 207
New in 4.0
PR 222
New in 4.0
See 2.2.3 fn:contains-subsequence
PR 250
New in 4.0
See 2.1.3 fn:foot
See 2.1.15 fn:trunk
PR 258
New in 4.0
PR 313
The second argument can now be a sequence of integers.
See 2.1.8 fn:remove
PR 314
New in 4.0
PR 326
Higher-order functions are no longer an optional feature.
See 1.2 Conformance
PR 419
New in 4.0
PR 434
New in 4.0
The function has been extended to allow output in a radix other than 10, for example in hexadecimal.
PR 482
Deleted an inaccurate statement concerning the behavior of NaN.
PR 507
New in 4.0
PR 546
It is no longer automatically an error if the input contains a codepoint that is not valid in XML. Instead, the codepoint must be a permitted character. The set of permitted characters is implementation-defined, but it is recommended that all Unicode characters should be accepted.
See 5.2.1 fn:codepoints-to-string
It is no longer automatically an error if the resource (after decoding) contains a codepoint that is not valid in XML. Instead, the codepoint must be a permitted character. The set of permitted characters is implementation-defined, but it is recommended that all Unicode characters should be accepted.
The rules regarding use of non-XML characters in JSON texts have been relaxed.
See 17.4.3 JSON character repertoire
It is no longer automatically an error if the input contains a codepoint that is not valid in XML. Instead, the codepoint must be a permitted character. The set of permitted characters is implementation-defined, but it is recommended that all Unicode characters should be accepted.
PR 631
New in 4.0
PR 662
Constructor functions now have a zero-arity form; the first argument defaults to the context item.
PR 680
The case-insensitive collation is now defined normatively within this specification, rather than by reference to the HTML "living specification", which is subject to change. The collation can now be used for ordering comparisons as well as equality comparisons.
PR 702
The function can now take any number of arguments (previously it had to be two or more), and the arguments can be sequences of strings rather than single strings.
See 5.4.4 fn:concat
PR 710
Changes the function to return a sequence of key-value pairs rather than a map.
PR 727
It has been clarified that loading a module has no effect on the static or dynamic context of the caller.
PR 828
The $predicate callback function accepts an optional position argument.
See 2.5.4 fn:filter
The $action callback function accepts an optional position argument.
The $predicate callback function now accepts an optional position argument.
The $action callback function now accepts an optional position argument.
PR 881
The way that fn:min and fn:max compare numeric values of different types has changed. The most noticeable effect is that when these functions are applied to a sequence of xs:integer or xs:decimal values, the result is an xs:integer or xs:decimal, rather than the result of converting this to an xs:double
See 2.4.5 fn:max
See 2.4.6 fn:min
PR 901
The optional third argument can now be supplied as an empty sequence.
The third argument can now be supplied as an empty sequence.
The second argument can now be an empty sequence.
The optional second argument can now be supplied as an empty sequence.
The 3rd, 4th, and 5th arguments are now optional; previously the function required either 2 or 5 arguments.
All three arguments are now optional, and each argument can be set to an empty sequence. Previously if $description was supplied, it could not be empty.
See 21.1.1 fn:error
The $label argument can now be set to an empty sequence. Previously if $label was supplied, it could not be empty.
See 21.2.1 fn:trace
PR 905
The rule that multiple calls on fn:doc supplying the same absolute URI must return the same document node has been clarified; in particular the rule does not apply if the dynamic context for the two calls requires different processing of the documents (such as schema validation or whitespace stripping).
See 17.1.1 fn:doc
PR 909
The function has been expanded in scope to handle comparison of values other than strings.
See 2.2.2 fn:compare
PR 924
Rules have been added clarifying that users should not be allowed to change the schema for the fn namespace.
See D Schemas
PR 925
The decimal format name can now be supplied as a value of type xs:QName, as an alternative to supplying a lexical QName as an instance of xs:string.
PR 932
The specification now prescribes a minimum precision and range for durations.
PR 933
When comments and processing instructions are ignored, any text nodes either side of the comment or processing instruction are now merged prior to comparison.
PR 940
New in 4.0
PR 953
Constructor functions for named record types have been introduced.
PR 962
New in 4.0
PR 969
New in 4.0
See 14.4.3 map:empty
PR 980
Atomic items of types xs:hexBinary and xs:base64Binary are now mutually comparable.
PR 984
New in 4.0
See 8.4.1 fn:seconds
PR 987
The order of results is now prescribed; it was previously implementation-dependent.
PR 1022
Regular expressions can include comments (starting and ending with #) if the c flag is set.
See 6.1 Regular expression syntax
See 6.2 Flags
PR 1028
An option is provided to control how the JSON null value should be handled.
PR 1032
New in 4.0
See 2.1.17 fn:void
PR 1046
New in 4.0
PR 1059
Use of an option keyword that is not defined in the specification and is not known to the implementation now results in a dynamic error; previously it was ignored.
See 1.7 Options
PR 1068
New in 4.0
PR 1072
The return type is now specified more precisely.
PR 1090
When casting from a string to a duration or time or dateTime, it is now specified that when there are more digits in the fractional seconds than the implementation is able to retain, excess digits are truncated. Rounding upwards (which could affect the number of minutes or hours in the value) is not permitted.
PR 1093
New in 4.0
PR 1117
The $options parameter has been added.
PR 1182
The $predicate callback function may return an empty sequence (meaning false).
See 2.5.3 fn:every
See 2.5.4 fn:filter
See 2.5.16 fn:some
PR 1191
The $options parameter has been added, absorbing the $collation parameter.
New in 4.0
PR 1250
For selected properties including percent and exponent-separator, it is now possible to specify a single-character marker to be used in the picture string, together with a multi-character rendition to be used in the formatted output.
PR 1257
The $options parameter has been added.
PR 1262
New in 4.0
PR 1265
The constraints on the result of the function have been relaxed.
PR 1280
As a result of changes to the coercion rules, the number of supplied arguments can be greater than the number required: extra arguments are ignored.
See 2.5.1 fn:apply
PR 1288
Additional error conditions have been defined.
PR 1296
New in 4.0
PR 1333
A new option is provided to allow the content of the loaded module to be supplied as a string.
PR 1353
An option has been added to suppress the escaping of the solidus (forwards slash) character.
PR 1358
New in 4.0
PR 1361
The term atomic value has been replaced by atomic item.
See 1.9 Terminology
PR 1393
Changes the function to return a sequence of key-value pairs rather than a map.
PR 1409
This section now uses the term primitive type strictly to refer to the 20 atomic types that are not derived by restriction from another atomic type: that is, the 19 primitive atomic types defined in XSD, plus xs:untypedAtomic. The three types xs:integer, xs:dayTimeDuration, and xs:yearMonthDuration, which have custom casting rules but are not strictly-speaking primitive, are now handled in other subsections.
See 23.1 Casting from primitive types to primitive types
The rules for conversion of dates and times to strings are now defined entirely in terms of XSD 1.1 canonical mappings, since these deliver exactly the same result as the XPath 3.1 rules.
See 23.1.2.2 Casting date/time values to xs:string
The rules for conversion of durations to strings are now defined entirely in terms of XSD 1.1 canonical mappings, since the XSD 1.1 rules deliver exactly the same result as the XPath 3.1 rules.
PR 1455
Numbers now retain their original lexical form, except for any changes needed to satisfy JSON syntax rules (for example, stripping leading zero digits).
PR 1473
New in 4.0
PR 1481
The function has been extended to handle other Gregorian types such as xs:gYearMonth.
See 9.5.1 fn:year-from-dateTime
See 9.5.2 fn:month-from-dateTime
The function has been extended to handle other Gregorian types such as xs:gMonthDay.
See 9.5.3 fn:day-from-dateTime
The function has been extended to handle other types including xs:time.
See 9.5.4 fn:hours-from-dateTime
See 9.5.5 fn:minutes-from-dateTime
The function has been extended to handle other types such as xs:gYearMonth.
PR 1504
New in 4.0
Optional $separator added.
PR 1523
New functions are provided to obtain information about built-in types and types defined in an imported schema.
New in 4.0
PR 1545
New in 4.0
PR 1565
The default for the escape option has been changed to false. The 3.1 specification gave the default value as true, but this appears to have been an error, since it was inconsistent with examples given in the specification and with tests in the test suite.
PR 1570
New in 4.0
PR 1587
New in 4.0
PR 1611
The spec has been corrected to note that the function depends on the implicit timezone.
See 2.2.2 fn:compare
PR 1671
New in 4.0.
PR 1703
Ordered maps are introduced.
Enhanced to allow for ordered maps.
See 14.4.7 map:find
See 14.4.14 map:put
The order of entries in maps is retained.
PR 1711
It is explicitly stated that the limits for $precision are implementation-defined.
See 4.4.6 fn:round
PR 1727
For consistency with the new function map:build, the handling of duplicates may now be controlled by supplying a user-defined callback function as an alternative to the fixed values for the earlier duplicates option.
PR 1734
In 3.1, given a mixed input sequence such as (1, 3, 4.2e0), the specification was unclear whether it was permitted to add the first two integer items using integer arithmetic, rather than converting all items to doubles before performing any arithmetic. The 4.0 specification is clear that this is permitted; but since the items can be reordered before being added, this is not required.
See 2.4.4 fn:avg
See 2.4.7 fn:sum
PR 1825
New in 4.0
PR 1856
Word boundaries can be matched. Lookahead and lookbehind assertions are supported. Assertions (including ^ and $) can no longer be followed by a quantifier.
See 6.1 Regular expression syntax
It is now permitted for the regular expression to match a zero-length string.
See 6.3.2 fn:replace
The output of the function is extended to allow the represention of captured groups found within lookahead assertions.
It is now permitted for the regular expression to match a zero-length string.
PR 1879
Additional options to control DTD and XInclude processing have been added.
PR 1897
The $replacement argument can now be a function that computes the replacement strings.
See 6.3.2 fn:replace
PR 1906
New in 4.0
See 14.5.10 fn:element-to-map-plan
New in 4.0.
PR 1910
An $options parameter is added. Note that the rules for the $options parameter control aspects of processing that were implementation-defined in earlier versions of this specification. An implementation may provide configuration options designed to retain backwards-compatible behavior when no explicit options are supplied.
See 17.1.1 fn:doc
PR 1991
Named record types used in the signatures of built-in functions are now available as standard in the static context.
PR 2001
New in 4.0.
PR 2013
Support for binary input has been added.
See 17.2.2 fn:parse-xml-fragment
New in 4.0
Support for binary input has been added.
New in 4.0
PR 2030
This description of the XSD validation process was previously found (with some duplication) in the XQuery and XSLT specifications; those specifications now reference this description. As a side-effects, the descriptions of the process in XQuery and XSLT are better aligned.
PR 2031
Introduced the concept of JNodes.
New in 4.0
See 16.1.1 fn:jtree
PR 2218
The rules for numeric comparison of mixed types are changed to be fully transitive.
PR 2223
An error may now be raised if the base URI is not a valid LEIRI reference.
PR 2224
The $action callback function now accepts an optional position argument.
PR 2228
New in 4.0
PR 2248
The specification now describes in more detail how to determine the effective encoding value.