XSLT and XQuery Serialization 4.0

1.1 Terminology

In this specification, where they are rendered in small capitals, the words must, must not, should, should not, may, required, and recommended are to be interpreted as described in [RFC2119].

[Definition: As is indicated in 12 Conformance, conformance criteria for serialization are determined by other specifications that refer to this specification. A serializer is software that implements some or all of the requirements of this specification in accordance with such conformance criteria.] A serializer is not required to directly provide a programming interface that permits a user to set serialization parameters or to provide an input sequence for serialization. In this document, material labeled as "Note" and examples are provided for explanatory purposes and are not normative.

Certain aspects of serialization are described in this specification as implementation-defined or implementation-dependent.

[Definition: Implementation-defined indicates an aspect that may differ between serializers, but whose actual behavior must be specified either by another specification that sets conformance criteria for serialization (see 12 Conformance) or in documentation that accompanies the serializer.]

[Definition: Implementation-dependent indicates an aspect that may differ between serializers, and whose actual behavior is not required to be specified either by another specification that sets conformance criteria for serialization (see 12 Conformance) or in documentation that accompanies the serializer.]

[Definition: In some instances, the input tree cannot be successfully converted into a sequence of octets given the set of serialization parameter (3 Serialization Parameters) values specified. A serialization error is said to occur in such an instance.] In some cases, a serializer is required to raise such an error. What it means to raise a serialization error is determined by the relevant conformance criteria (12 Conformance) to which the serializer conforms. In other cases, there is an implementation-defined choice between raising a serialization error and performing a recovery action. Such a recovery action will allow a serializer to produce a sequence of octets that might not fully reflect the usual requirements of the parameter settings that are in effect.

[Definition: Where this specification indicates that two strings are to be compared without regard to case, the serializermust translate any characters in the range U+0041 (LATIN CAPITAL LETTER A, A) through U+005A (LATIN CAPITAL LETTER Z, Z) inclusive, to the corresponding lower-case letters in the range U+0061 (LATIN SMALL LETTER A, a) through U+007A (LATIN SMALL LETTER Z, z) only for the purposes of making the comparison. The comparison succeeds if the two strings are the same length and the code point of each character in the first string is equal to the code point of the character in the corresponding position in the second string.]

Many terms used in this document are defined in the XPath specification [XPath 4.0] or the Data Model specification [XDM 4.0]. Particular attention is drawn to the following:

• [Definition: The term atomization is defined in [XPath 4.0] section 2.6.3 Atomization.]

• [Definition: The term node is defined in [XPath 4.0] section 2 Basics. There are seven kinds of nodes in the data model: document, element, attribute, text, namespace, processing instruction, and comment.]

• [Definition: The term sequence is defined in [XPath 4.0] section 2 Basics. A sequence is an ordered collection of zero or more items.]

• [Definition: The term function item is defined in [XDM 4.0] section 8.1 Function Items.]

• [Definition: The term map item is defined in [XDM 4.0] section 8.2 Map Items.]

• [Definition: The term array item is defined in [XDM 4.0] section 8.3 Array Items.]

• [Definition: The term string is defined in [XDM 4.0] section 4.1.5 XML and XSD Versions.]

• [Definition: The term character is defined in [XDM 4.0] section 4.1.5 XML and XSD Versions.]

• [Definition: The term codepoint is defined in [XDM 4.0] section 4.1.5 XML and XSD Versions.]

• [Definition: The term string value is defined in [XDM 4.0] section 7.6.12 string-value Accessor. Every node has a string value. For example, the string value of an element is the concatenation of the string values of all its descendant text nodes.]

• [Definition: The term expanded QName is defined in [XPath 4.0] section 2 Basics. An expanded QName consists of an optional namespace URI and a local name. An expanded QName also retains its original namespace prefix (if any), to facilitate casting the expanded QName into a string.]

• [Definition: An expanded-QName whose namespace part is an empty sequence, or an element or attribute whose name expands to such an expanded-QName, is referred to as having a null namespace URI].[Definition: An expanded-QName whose namespace part is the empty sequence, or an element or attribute whose name expands to such an expanded-QName, is referred to as having a null namespace URI].[Definition: An expanded-QName whose namespace part is anthe empty sequence, or an element or attribute whose name expands to such an expanded-QName, is referred to as having a null namespace URI].

• [Definition: An element or attribute that does not have a null namespace URI, is referred to as having a non-null namespace URI].

• [Definition: A space character, TAB character, CR character or NL character is referred to as a whitespace character.]

Where this specification indicates that an XSLT instruction is evaluated, the behavior is as specified by [XSLT 4.0]. Where it indicates that an XQuery expression is evaluated, the behavior is as specified by [XQuery 4.0].

3.1 Setting Serialization Parameters by Means of a Parameter Document

A host languagemay provide, by reference to this section, a mechanism by which the settings of serialization parameters are supplied in the form of an output:serialization-parameters element node.A host languagemay provide, by reference to this section, a mechanism by which the settings of serialization parameters are supplied in the form of an output:serialization-parameters element node.A host languagemay provide, by reference to this section, a mechanism by which the settings of serialization parameters are supplied in the form of an output:serialization-parameters element node.

[Definition: An output:serialization-parameters element node used to hold the settings of serialization parameters is referred to as a parameter document].

The parameter documentmust be processed as if by the procedure described below.

With the exception of the use-character-maps parameter, the setting of each serialization parameter defined in this specification is equal to the result of evaluating the XQuery expression

document { . } 
   /output:serialization-parameters
   /(validate lax { 
      output:*[local-name() eq $param-name] 
   })
   /data(@value)

or equivalently the XSLT instructions

<xsl:sequence>
  <xsl:variable name="validated-instance">
    <xsl:document validation="lax">
      <xsl:sequence select="
        self::output:serialization-parameters
        /output:*
       [local-name() eq $param-name]"/>
    </xsl:document>
  </xsl:variable>
  <xsl:sequence select="$validated-instance
                        /data(@value)"/>
</xsl:sequence>

with the parameter document as the context value, the param-name variable bound to a value of type xs:string equal to the local part of the name of the particular serialization parameter, and the other components of the dynamic context and static context as specified in the subsequent tables. If in any case evaluating this expression would yield an error, serialization error [err:SEPM0017] results.

If the result of evaluating this expression for a particular serialization parameter is the empty sequence, then

1. If the parameter is either cdata-section-elements or suppress-indentation, and the result of evaluating the XQuery expression

   document { . }
   /output:serialization-parameters
   /(validate lax {  
      output:*[local-name() eq $param-name]
   })

   or equivalently the XSLT instructions

   <xsl:sequence>
     <xsl:variable name="validated-instance">
       <xsl:document select="." validation="lax">
         <xsl:sequence select="
           self::output:serialization-parameters
           /output:*
          [local-name() eq $param-name]"/>
       </xsl:document>
     </xsl:variable>
     <xsl:sequence select="$validated-instance"/>
   </xsl:sequence>

   with the same settings of the static context and dynamic context is not anthe empty sequence, the setting of the parameter is the empty list;

2. otherwise, the setting of the parameter is absent.

The components of the static context used in evaluating the XQuery expressions or XSLT instructions are as defined in the following table.

The remaining components of the dynamic context used in evaluating the XQuery expressions or XSLT instructions in the preceding table are as defined in the following table.

In the case of the use-character-maps parameter, the XQuery expression

document { . }
/output:serialization-parameters
/ ( validate lax { output:use-character-maps } )
/output:character-map[@character eq $char]
/string(@map-string)

or equivalently the XSLT instructions

<xsl:sequence>
  <xsl:variable name="validated-instance">
    <xsl:document validation="lax">
      <xsl:sequence select="
        self::output:serialization-parameters
         /output:use-character-maps"/>
    </xsl:document>
  </xsl:variable>
  <xsl:sequence select="$validated-instance                          
                        /output:character-map
                        [@character eq $char]
                        /string(@map-string)"/>
</xsl:sequence>

is evaluated for each Unicode character that is permitted in an XML document. The dynamic context and static context used to evaluate the expression are as defined above, except that the component In-scope variables is the set {char} and the value of the variable "char" is a value of type xs:string of length one whose value is the Unicode character under consideration. If the result of evaluating the expression is not an empty sequence, the pair consisting of the Unicode character and the result of evaluating the expression is part of the list of pairs in the value of the use-character-maps parameter. It is a serialization error [err:SEPM0018] if the result of evaluating this expression for any character is a sequence of length greater than one.is evaluated for each Unicode character that is permitted in an XML document. The dynamic context and static context used to evaluate the expression are as defined above, except that the component In-scope variables is the set {char} and the value of the variable "char" is a value of type xs:string of length one whose value is the Unicode character under consideration. If the result of evaluating the expression is not the empty sequence, the pair consisting of the Unicode character and the result of evaluating the expression is part of the list of pairs in the value of the use-character-maps parameter. It is a serialization error [err:SEPM0018] if the result of evaluating this expression for any character is a sequence of length greater than one.is evaluated for each Unicode character that is permitted in an XML document. The dynamic context and static context used to evaluate the expression are as defined above, except that the component In-scope variables is the set {char} and the value of the variable "char" is a value of type xs:string of length one whose value is the Unicode character under consideration. If the result of evaluating the expression is not anthe empty sequence, the pair consisting of the Unicode character and the result of evaluating the expression is part of the list of pairs in the value of the use-character-maps parameter. It is a serialization error [err:SEPM0018] if the result of evaluating this expression for any character is a sequence of length greater than one.

Using the same settings of the components of the dynamic context and static context, serialization error [err:SEPM0019] results if the result of evaluating the following XQuery expression is not true

(document { . })/output:serialization-parameters
   /(count(distinct-values(*/node-name(.))) eq (count(*)))

or equivalently if the result of evaluating the following XSLT instructions is not true.

<xsl:sequence>
  <xsl:variable name="doc">
    <xsl:document>
      <xsl:sequence select="."/>
    </xsl:document>
  </xsl:variable>
  <xsl:sequence
    select="$doc/output:serialization-parameters
                /(count(distinct-values(
                    */node-name(.))) 
                eq (count(*)))"/>
</xsl:sequence>

The result of evaluating either will be false if the parameter document supplies a value for any particular serialization parameter more than once, or will be the empty sequence if the parameter document is not an element node whose local name is serialization-parameters and whose namespace URI is http://www.w3.org/2010/xslt-xquery-serialization.

The host languagemay provide additional mechanisms for overriding the values of any serialization parameters specified through the mechanism defined in this section, as well as additional mechanisms for specifying the values of any serialization parameters whose values are absent after applying the mechanism defined in this section.

If the parameter document contains elements or attributes that are in a namespace other than http://www.w3.org/2010/xslt-xquery-serialization, the implementation may interpret them to specify the values of implementation-defined serialization parameters in an implementation-defined manner.

<output:serialization-parameters 
    xmlns:output 
    = "http://www.w3.org/2010/xslt-xquery-serialization">
  <output:method value="xml"/>
  <output:version value="1.0"/>
  <output:indent value="yes"/>
</output:serialization-parameters>

<output:serialization-parameters
    xmlns:output
    = "http://www.w3.org/2010/xslt-xquery-serialization"
    xmlns:book="http://example.org/book"
    xmlns="http://example.org/book/chapter">
  <output:cdata-section-elements value="heading book:footnote"/>
</output:serialization-parameters>

<output:serialization-parameters
    xmlns:output
    = "http://www.w3.org/2010/xslt-xquery-serialization"
    xmlns="http://example.org/ext">
  <output:method value="html"/>
</output:serialization-parameters>

<output:serialization-parameters
    xmlns:output
    = "http://www.w3.org/2010/xslt-xquery-serialization"
    xmlns:ext="http://example.org/ext">
  <output:method value="ext:jsp"/>
  <output:use-character-maps>
    <output:character-map character="&#xAB;" map-string="&lt;%"/>
    <output:character-map character="&#xBB;" map-string="%&gt;"/>
  </output:use-character-maps>
</output:serialization-parameters>

7.4 Writing Comments and Processing Instructions

Comment nodes are written as HTML comments, using the syntax <!--comment-->.

Prior to HTML5, the HTML output method must terminate processing instructions with > rather than ?>. It is a serialization error [err:SERE0015] to use the HTML output method when > appears within a processing instruction in input tree.

With HTML5, a processing instruction must be written as an HTML comment, in the form <!--?targetdata?-->. Here target is the name of the processing instruction, and data is its string value; they are separated by a single space. If the string value of the processing instruction contains two adjacent hyphens, they must be separated in the output by a single space. The space separator is omitted if the string value of the processing instruction is zero-length. For example, the processing instruction <?xml-stylesheet type="text/xsl" href="render.xsl"?> is serialized as <--?xml-stylesheet type="text/xsl" href="render.xsl"?-->With HTML5, a processing instruction must be written as an HTML comment, in the form <!--?targetdata?-->. Here target is the name of the processing instruction, and data is its string value; they are separated by a single space, but this omitted in the case where data is zero-length. If either the name or the string value of the processing instruction contains two adjacent hyphens, they must be separated in the output by a single space. For example, the processing instruction <?xml-stylesheet type="text/xsl" href="render.xsl"?> is serialized as <--?xml-stylesheet type="text/xsl" href="render.xsl"?-->With HTML5, a processing instruction must be written as an HTML comment, in the form <!--?targetdata?-->. Here target is the name of the processing instruction, and data is its string value; they are separated by a single space, but this omitted in the case where data is zero-length. If theeither the name or the string value of the processing instruction contains two adjacent hyphens, they must be separated in the output by a single space. The space separator is omitted if the string value of the processing instruction is zero-length. For example, the processing instruction <?xml-stylesheet type="text/xsl" href="render.xsl"?> is serialized as <--?xml-stylesheet type="text/xsl" href="render.xsl"?-->

9.1 Serializing Items

An individual item is serialized as follows:

• A JNode is serialized by serializing its ·content· property.

• An array item in the input tree is serialized to a JSON array by outputting the serialized JSON value of each member within the array separated by delimiters according to the JSON array syntax, i.e. [member, member, ...]. Each member in the array is to be serialized by recursively applying the rules in this section.

• A map item in the input tree is serialized to a JSON object by outputting, for each key/value pair, the string value of the key to a JSON string, followed by the serialized JSON value of the entry, separated by delimiters according to the JSON object syntax, i.e. {key:value, key:value, ...}. The key/value pairs in the serialized output retain the entry order^DM of entries in the map, unless canonical is true, in which case map entries must be sorted according to the rules of [RFC8785].

  Note:

  These rules require sorting according to the UTF-16 representation of the string, which is not the same (in the presence of surrogate pairs) as sorting using the XPath codepoint collation.

  If any two keys of the map item have the same string value, serialization error [err:SERE0022] is raised, unless the allow-duplicate-names parameter is true and the canonical parameter is false.

• A node in the input tree is serialized to a JSON string by outputting the result of serializing the node using the method specified by the json-node-output-method parameter. If the value of the canonical parameter is false, the node is serialized with the serialization parameter omit-xml-declaration set to true and with no other serialization parameters set, otherwise (canonical is true) all parameters are inherited by the sub-serialization process.

• An atomic value^XP of type xs:numeric is serialized to a JSON number.

  If the canonical parameter is true, then the value is cast to type xs:double and the result is serialized according to the rules of [RFC8785] (and by extension [RFC7493]).

  If the canonical parameter is false then:

  • A value of type xs:decimal (including xs:integer) is output in the format that results from casting the value to xs:string.

  • A value of type xs:float is first cast to type xs:double, and then output in the same way as an xs:double.

  • Implementations may serialize an xs:double value using any lexical representation of a JSON number defined in [RFC 7159], but it is recommended to use the same representation as when the canonical parameter is true.

  • The value NaN is serialized as the JSON token null.

  • The values positive and negative infinity are serialized as 1e9999 and -1e9999 respectively.

    Note:

    These values are permitted by the JSON grammar, but some JSON parsers may reject them. For interoperability, they should be avoided.

• An atomic item^XP of type xs:boolean is serialized to the JSON token true or false.

• An atomic item^XP of type xs:QName in the input tree whose namespace part is "http://www.w3.org/2005/xpath-functions" and whose local part is "null" is serialized to the JSON token null.

  Note:

  This rule is introduced in 4.0, along with an option in the fn:parse-json function to allow a user-defined representation of the JSON value null. While the default representation of null as an empty sequence is usable in many circumstances, an explicit representation of null as a recognizable item can make some operations on JSON-derived values easier.This rule is introduced in 4.0, along with an option in the fn:parse-json function to allow a user-defined representation of the JSON value null. While the default representation of null as the empty sequence is usable in many circumstances, an explicit representation of null as a recognizable item can make some operations on JSON-derived values easier.This rule is introduced in 4.0, along with an option in the fn:parse-json function to allow a user-defined representation of the JSON value null. While the default representation of null as anthe empty sequence is usable in many circumstances, an explicit representation of null as a recognizable item can make some operations on JSON-derived values easier.

• Any other atomic value^XP in the input tree is serialized to a JSON string by outputting the result of applying the fn:string function to the item.

• An empty sequence in the input tree is serialized to the JSON token null.

• A sequence of length greater than one in the input tree will result in a serialization error [err:SERE0023].

• Any item in the input tree of a type not specified in the above list will result in a serialization error [err:SERE0021].

[Definition: Whenever a value is serialized to a JSON string, the following procedure is applied to the supplied string:]

1. If the canonical parameter is true, then the string is output as specified in [RFC8785]. The string is first normalized if requested using the normalization-form parameter. The serialization parameters escape-solidus and use-character-maps are ignored.

   Note:

   The escaping rules are restated here for convenience:

   • If the Unicode value falls within the traditional ASCII control character range (U+0000 (NULL) through U+001F (IS1) ), it must be serialized using lowercase hexadecimal Unicode notation (\uhhhh) unless it is in the set of predefined JSON control characters U+0008 (BACKSPACE) , U+0009 (TAB) , U+000A (NEWLINE) , U+000C (FORM FEED) , or U+000D (CARRIAGE RETURN) , which must be serialized as \b, \t, \n, \f, and \r, respectively.

   • If the Unicode value is outside of the ASCII control character range, it must be serialized “as is” unless it is equivalent to U+005C (REVERSE SOLIDUS, BACKSLASH, \) or U+0022 (QUOTATION MARK, ") , which must be serialized as \\ and \", respectively.

   Note that the C1 control characters (codepoints 127-159) are not included in this list.

   Note:

   In canonical JSON, property values within an object are sorted based on their UTF16 representation. This corresponds to the default sort order in languages such as Javascript, Java, and C#, but in the presence of codepoints above 65535, it is not the same as the order produced by Unicode codepoint collation.

2. Otherwise (when canonical is false):

   1. Any character in the string for which character mapping is defined (see 11 Character Maps) is substituted by the replacement string defined in the character map.

   2. Any other character in the input string (but not a character produced by character mapping) is a candidate for Unicode Normalization if requested by the normalization-form parameter, and JSON escaping. JSON escaping replaces the characters U+005C (REVERSE SOLIDUS, BACKSLASH, \) , U+0008 (BACKSPACE) , U+000C (FORM FEED) , U+000A (NEWLINE) , U+000D (CARRIAGE RETURN) , U+0009 (TAB) , or U+005C (REVERSE SOLIDUS, BACKSLASH, \) by the corresponding JSON escape sequences \", \b, \f, \n, \r, \t, or \\ respectively, and any other codepoint in the range 1-31 or 127-159 by an escape in the form \uHHHH where HHHH is the hexadecimal representation of the codepoint value. Escaping further replaces the solidus character (/) by the escape sequence \/ if the escape-solidus parameter is set to true, but not if it is set to false. Escaping is also applied to any characters that cannot be represented in the selected encoding.

3. The resulting string is enclosed in double quotation marks.

Finally, encoding, as controlled by the encoding parameter, converts the character stream produced by the preceding rules into an octet stream. The encoding parameter is ignored if canonical is true.

9.2 JSON Lines

When the json-lines parameter is set to true, the serialized output is written in json-lines format, as defined in [JSON Lines].

If json-lines is set to true, then each item in the input value is serialized indepedently as a JSON text, and the resulting serializations are then concatenated using a single U+000A (NEWLINE) character as a separator.

If json-lines and indent are both set to true, then the serialization of each individual item may include added U+0020 (SPACE) and U+0009 (TAB) characters for formatting purposes, but it must not include U+000A (NEWLINE) or U+000D (CARRIAGE RETURN) characters.

The json-lines specification allows U+000D (CARRIAGE RETURN) characters to appear, and does not treat them as significant; however, they are likely to cause practical problems. For example processing such input using the XPath expression unparsed-text-lines($uri) ! parse-json() would fail. The serializer must not output any U+000D (CARRIAGE RETURN) characters, either at the end of a line or elsewhere.

The json-lines specification allows a terminating U+000A (NEWLINE) character after the last line. In the interests of interoperability, however, the serializer must not output such a terminator.

If the input value is an empty sequence, then it is serialized as a zero-length string (rather than as the string null).If the input value is the empty sequence, then it is serialized as a zero-length string (rather than as the string null).If the input value is anthe empty sequence, then it is serialized as a zero-length string (rather than as the string null).

The item-separator parameter has no effect.