@qt4cg statuses

The existing XQFO rules are pretty heterogeneous when it comes to presenting the handling of default values.

Sometimes, they repeat what the signature says…

https://qt4cg.org/specifications/xpath-functions-40/Overview.html#func-name

fn:name(
  $node  as node()?  := .
) as xs:string

If the argument is omitted, it defaults to the context value (.).

Sometimes, it is further elaborated…

https://qt4cg.org/specifications/xpath-functions-40/Overview.html#func-string

fn:string(
  $value  as item()?  := .
) as xs:string

In the zero-argument version of the function, $value defaults to the context value. That is, calling fn:string() is equivalent to calling fn:string(.).

…sometimes, it is ignored:

https://qt4cg.org/specifications/xpath-functions-40/Overview.html#func-upper-case

fn:upper-case(
  $value  as xs:string?	
) as xs:string

If $value is the empty sequence, the zero-length string is returned. […]

I would propose to revise the rules, and to consistently remove phrases on choosing default values. I believe this will make the rules better readable (once you have understood what default values are, there should be no need explaining it again and again). If we want to keep the information, though, we should add it everywhere.

I can make an attempt if no one objects.

(Note: Edited to show that the parses though XNodePattern are subtly different in the two cases - I originally hadn't spotted the difference.)

I think there is a (minor?) ambiguity in the current XSLT pattern grammar, involving the the 'shortened' type match patterns such as map(*), item(), record(...) and enum("red").

If the type is enclosed in the type(...) qualifier, e.g. type(map(*)) then the grammar appears to be unambiguous, yielding a parse down through pattern productions to SimpleNodeTest to a TypeTest/SequenceType:

<Pattern>
 <UnionPattern>
  <IntersectExceptPattern>
   <PrimaryPattern>
    <XNodePattern>
     <PathExprP>
      <RelativePathExprP>
       <StepExprP>
        <AxisStepP>
         <ForwardStepP>
          <AbbrevForwardStepP>
           <SimpleNodeTest>
            <TypeTest>
             <SequenceType>
              <ItemType>
               <RegularItemType>
                <MapType>
                 <AnyMapType/>

but if the 'short form' of the pattern is used, viz map(*), then there are two parse routes. The first is close to that given above, but not exactly the same:

<Pattern>
 <UnionPattern>
  <IntersectExceptPattern>
   <PrimaryPattern>
    <XNodePattern>
     <PathExprP>
      <RelativePathExprP>
       <StepExprP>
        <AxisStepP>
         <ForwardStepP>
          <AbbrevForwardStepP>
           <SimpleNodeTest>
            <TypeTest>
             <RegularItemType>
              <MapType>
               <AnyMapType/>

where an alternative of TypeTest/RegularItemType (without the type(..) surround) has been chosen.

The second is:

<Pattern>
 <UnionPattern>
  <IntersectExceptPattern>
   <PrimaryPattern>
    <TypePattern>
     <MapType>
      <AnyMapType/>

where PrimaryPattern parses through TypePattern, and perhaps might be the intended outcome. The same logic applies to any of the 'shortened' type match instances.

Whether this is significant or not, I'm not sure. A cursory inspection of the spec. doesn't show any suggestion of preference for the latter form.

As TypeTest is defined in XPath and can of course be used in predicates and selectors, it may be difficult to avoid this ambiguity from the XSLT pattern productions. Will need to think about alternatives.

1. Adds xsl:record, xsl:map-entry, xsl:array-member to the list of instructions that inhibit creating of an implicit document node. (It makes no sense to wrap an array or map in a document node)
2. Mentions the revised rules in places where one would expect it to be mentioned.

1. Corrects the statement that element(N, T) will only match a schema-validated element, which is not true if T is xs:anyType or xs:untyped
2. Adds notes and example, in both XPath and XSLT, to make clear the relationship between element(N), element(N, xs:anyType), and element(N, xs:anyType?)

In XPath §3.2.7.2 we say

Where a [TypeName] is included in an [ElementTest] T, an element node will only match the test if it has been validated against a schema that defines type T; furthermore, T must be present in the [in-scope schema definitions] of the static context of the [ElementTest].

This obviously doesn't apply where T is xs:untyped.

And I don't think it applies where T is xs:anyType. An untyped element matches element(E, xs:anyType) by virtue of the fact that xs:untyped is a subtype of xs:anyType.

I think the rules in XPath §3.3.2.5.3 say that element(E) and element(E, xs:anyType?) are essentially the same type (each is a subtype of the other). However, element(E) and element(E, xs:anyType) are NOT the same type, because the former allows the element to be nilled.

In the XSLT rules for patterns (which is how I got here), element(E), element(E, xs:anyType), and element(E, xs:anyType?) are all valid patterns, and their meaning is defined by reference to XPath item types. However,

• element(E) and element(E, xs:anyType?) have different default priorities even though they match exactly the same nodes
• element(E, xs:anyType) and element(E, xs:anyType?) have the same default priority even though the first is a supertype of the second

This is all unchanged from XSLT 3.0

This is mostly a copy of #2290, translated into the RELAX NG grammar.

I think the grammar still needs review, but I’m trying to keep it up-to-date with the XSD in the meantime.

We could add an options parameter to the XSLT document function, providing the same options as fn:doc. It's complicated by the fact that fn:document already has an optional second argument (a node whose base URI is to be used for resolving relative URIs) -- but we could solve that as we have done with deep-equal, by incorporating the base-uri-node into a new options parameter and making the second argument have a type of (node()|map(*)).

Or we could leave the functionality of fn:document alone, treating it as legacy, but add to the rules to define its effect in terms of the doc() function. We could mandate the values of fn:doc() options such as DTD-validation, schema-validation, space stripping, etc, or we could leave these implementation-defined.

Currently XSLT §4.3 says that documents loaded using doc(), document(), or collection() are subject to preprocessing: stripping of whitespace and type annotations as controlled by declarations in the stylesheet. It's a bit weird that XSLT should interfere with the semantics of doc() and collection() in this way, and now that doc() (and in future collection()?) have options parameters to control some of these behaviours, it's not clear how they are supposed to interact. One approach would be to say that if doc() or collection() has an options parameter, this overrides any default processing defined in §4.3.

Once a module is loaded its parameters, at least its loading-parameters like its location, should be fixed an not be able to be changed. The use of a 'load-xquery-module"-function, however, creates the possibility of specifiing contradicting loading-parameters.

My suggestion: Don't let the use of the 'location-hints'-parameter of the 'load-xquery-module' be 'implementation-defined'. Let these parameters formally be 'ignored', and therefor optional, if a module with namespace: '$module-uri', is already in memmory.

Background: My xquery compiler always requiers an 'location-hints'-parameter ( of course, if the 'content'-parameter is omitted) even if the module is already loaded. At the moment this is formally a legit choice. However it makes it unnecesary tedious to, for instance, link XML-elements to modules by means of its tagname: I unnecesary need the module-location where it is not available.

Importance: This is important because the possibilyty of contradiction should always, for quality sake, be avoided. And, dynamicly linking XML-elements to modules would open the possibility of polymorfism on XML-elements which, I think, would be an awesom feature.

Updated schema for XSLT 4.0 stylesheets (verified against the current test cases using test catalog-005).

Changes include: xsl:record, xsl:array, xsl:select, @schema-role, mode/@copy-namespaces, EQName syntax, xsl:function/@name

This PR contains some more editorial changes. Notes:

• I have revised the »Summary of Changes« to (hopefully) make it more clear which »arrows« the description refers to, and I have updated the icons.
• I have corrected some code equivalencies in the Binary Module. The bin:shift equivalency (possibly others) still needs to be fixed).

XSLT §9.3 states that xsl:variable does not create an implicit document node if the sequence constructor includes an xsl:map, xsl:array, or xsl:select instruction.

1. The list should also include xsl:record, xsl:map-entry, xsl:array-member (anything that will never construct an XNode and therefore where wrapping in a document node is doomed to failure).
2. The rule needs to be restated in §9.4 (creating implicit document nodes)

Something about the proposed change from unordered to ordered maps in QT4 has been bothering me for a long time. I hadn't been able to put my finger on it until whilst listening to John and Juri talk about QT4 at Declarative Amsterdam 2025, I think I now understand what at least one of the alarms going off in my head was trying to tell me.

I'd just like to say that if the below is not a problem and I have misunderstood the explanation of QT4, many apologies! I would love to hear how this isn't a problem, but otherwise...

I fear that the switch now from Unordered Maps in XDM 3.1 to Ordered Maps in XDM 4.0 breaks JSON interoperability.

I recall that when we were working on XQuery 3.1 the major use-case was interoperability with JSON. It says as much in the XQuery 3.1 spec here:

XQuery 3.1 extends XQuery to support JSON as well as XML, adding maps and arrays to the data model

In XQuery 3.1, maps are unordered. In JSON, these things are not called "maps", they are called "objects". If we look at the various JSON specifications, we can see that Objects (or Maps as they are known in XDM) are unordered:

1. From json.org:

An object is an unordered set of name/value pairs.

2. From IETF - RFC 7159 - The JavaScript Object Notation (JSON) Data Interchange Format

   1. Section 1:

   An object is an unordered collection of zero or more name/value pairs

   2. Section 4:

   JSON parsing libraries have been observed to differ as to whether or not they make the ordering of object members visible to calling software. Implementations whose behavior does not depend on member ordering will be interoperable in the sense that they will not be affected by these differences.

So, I think we can all agree that:

1. Object members are unordered in JSON.
2. Their equivalent in XDM, Maps, were unordered in version 3.1, but they are scheduled to become ordered in version 4.0.

So, what's the problem that I foresee with making Maps ordered?

The problem is not Maps in isolation, if you took XQuery 3.1 and simply changed Maps from Unordered to Ordered, I can't foresee any breakage or interoperability problems with JSON.

The problem that I foresee right now in QT4, that I think I can show breaks JSON interoperability, is the use of Ordered Maps to provide sibling access to members of a map. There may be other areas of the spec that have similar problems too, but I haven't given them any consideration yet.

---

Example 1

Given the following JSON document (doc1.json):

[
	{
		"name": "Adam",
		"age": 44,
		"location": "Bardonecchia"
	},
	{
		"name": "Bob",
		"years": 33,
		"location": "London"
	}
]

The following QT4 XQuery:

fn:json-doc("doc1.json")//name/following-sibling::*[1]

I can't find a working implementation to test this against, but if I have understood the QT4 spec, then running the above QT4 XQuery would yield something like:

(44, 33)

Example 2

The same QT4 XQuery from above, but instead executed against this JSON document (doc2.json):

[
	{
		"name": "Adam",
		"age": 44,
		"location": "Bardonecchia"
	},
	{
		"name": "Bob",
		"location": "London",
		"years": 33
	}
]

I think would yield something like:

(44, "London")

---

The problem as I see it is that both of those two JSON documents (doc1.json) and (doc2.json) are from a JSON perspective canonically identical, yet our XQuery which hasn't changed, produces different results against each JSON document.

As systems (e.g. Web APIs) that produce JSON are not required to enforce an ordering of object members. Calling the same application or API that produces a JSON document more than once (standard when talking to Web APIs), may validly produce canonically identical JSON documents even if their object members are in a different order.

IMHO we simply cannot rely on an ordering of Map members within XDM for sibling lookups. If we allow users to program against JSON documents in QT4 using the Sibling Axis (over Ordered Maps), they will get bitten and bitten hard when their XQuery starts returning different or no results because the ordering in their JSON changed (but canonically is the same).

I understand that there is a general desire to enable XPath Axis to operate over JSON documents, and I think that's a wonderful thing in general. However, if I have understood what it means for Maps and Sibling Axis, I think some changes have to be made before we create a big sibling shaped hole for users to fall into.

Closes #2279

Fix #2198

Adds a new pi-for-cdata parameter allowing arbitrary addition of CDATA sections to serialized XML output, by outputting the relevant text as a processing instruction node with a chosen name.

This PR contains the necessary changes to the serialization spec; I would like a go-ahead from the CG before doing the (somewhat mechanical) work to add the necessary changes to the other specs.

I chose this approach in preference to the mooted conditional-data-elements approach because it is a lot more flexible. (Some badly written XML interfaces require precise control over CDATA generation: we might disapprove, but the requirement exists.)

Now that we have dropped binary types from fn:parse-csv, we need to fix the rules of fn:csv-doc, specifically:

The effect of the two-argument function call fn:csv-doc($H, $M) is equivalent to the function composition fn:unparsed-binary($H) => fn:parse-csv($M).

Next, we should re-align the rules of fn:csv-doc and fn:json-doc. The detection of the encoding should be clarified, possibly by referring to the rules proposed in #2282, or the current version of bin:decode-string.

Changes the rule for extension attributes, instead of saying they MUST NOT cause non-conformant behaviour, we now say SHOULD NOT, and give examples where it might be appropriate to break this rule, e.g. for security. Also, extension attributes can now modify the behaviour of the serializer in any way they like.

The PR also moves the section on Extension Attributes to a more logical place in the spec.

This PR separates the operation of examining a string to determine its encoding and the presence of a BOM, from the operation of decoding octets. The purpose is to simplify the decode-string function and to reduce complex interactions between its arguments, especially in cases where the BOM might be at the start of a message and the strings to be decoded at some later position. It also has a better chance of retaining compatibility with 1.0 implementations (though it's hard to guarantee that since 1.0 was underspecified).

Fix #2280

This PR revises the spec of xsl:array and xsl:array-member in the light of experience writing test cases.

The zero-arity function item constructed by xsl:array-member now carries a function annotation which means that it can be unambiguously recognized as having been so constructed. This allows better error checking and diagnostics, and it also allows xsl:array to be more tolerant of using xsl:array-member to create singleton array members in circumstances where it is not strictly required.

Writing tests for the new specification of xsl:array, it's tricky to remember when to use xsl:array-member and when not; and the diagnostics for getting it wrong are obscure (typically, if you use xsl:array-member when it's not needed, you end up with an array of functions, and this leads to subsequent errors saying a function was supplied when something else was expected).

I think the answer to this may be:

(a) the zero-arity function delivered by xsl:array-member should carry the function annotation %xsl:array-member

(b) when xsl:array sees a zero-arity function with this annotation in the member sequence, it should invoke the function; any other item (including any other function) is passed through unchanged.

The effect is that any singleton item delivered by the content expression for xsl:array is treated as a single-item member of the constructed array, with the exception of a singleton item constructed using the xsl:array-member instruction, which is treated as a sequence-valued member.