@qt4cg statuses

I wonder what we could do to get rid of the cumbersome ! jvalue() invocations. The current JMESPath tests have 290 of them alone, most of which cannot be avoided. In the JSONPath tests, I could only circumvent them by adding jtree(...) wrappers in the test assertions.

We could let the attribute axis mean “value of the child JNode”:

$in/foo ! { "bar": @bar, "baz": @baz }    (: bar ! jvalue(), baz ! jvalue() :)
array { @* }                              (: array { * ! jvalue() } :)
sort($in/numbers/@*)                      (: values, sorted :)
$in/foo/*[exists(@key)]                   (: key exists and is not null :)
$in/foo/*[deep-equal(@key, [ 0 ])]

Another syntax extension could be a new “value” axis: value::*.

Someone suggested /?, but I assume it would result in ambiguities ($in/foo ! jvalue() → $in/foo/?)

A more radical approach would be to ignore JNode/XNode boundaries. The following query…

let $nodes := { 'nodes': [ <xml><w/></xml> ] }/nodes
return jvalue($nodes/1)//w

…could then be simplified to:

{ 'nodes': [ <xml><w/></xml> ] }/nodes/1//w

I haven’t reflected on the implications; suggestions are welcome.

We may need to revise the IETF MIME type registrations for XSLT and XQuery, in particular to update the security sections.

In the meantime, it might be a good idea to remove Appendix F from the XQuery specification: this is a copy of the old MIME type registration that now needs updating. (For XSLT, this has already been done.)

The xsl:source-document instruction and stream-available should have options analogous to those on fn:doc and fn:document.

Similarly xsl:merge-source.

The function file:resolve-path contradicts the file paths rules:

• “If $path is an absolute path, it is not changed” conflicts with “all paths are normalized […] with operating-system-specific directory separators”. • The Windows examples return forward slashes ('C:/Windows/INF/'); a trailing separator is only appended for existing directories, although for $base it decides “directory”.

I think we should return absolute paths normalized and preserve a trailing (possibly normalized) separator. Examples:

file:resolve-path('C:/nonexist/')              (: C:\nonexist; proposed: C:\nonexist\ :)
file:resolve-path('sub/', 'C:/Temp/claude/')   (: C:\Temp\claude\sub; proposed: C:\Temp\claude\sub\ :)

Fix #2991

Marks functions that access external resources as security-sensitive; adds the concept that these are evaluated in sandbox mode if the function is referenced in a (statically) untrusted context; defines the rules for each security-sensitive function when executed in sandboxed mode.

Still to do: similar treatment of functions defined in XSLT and the File module; extra words in the XQuery spec on import module and import schema, and similar constructs in XSLT, and xsl:evaluate.

By rewriting the queries of the JSONPath and JMESPath test suites to our JNode syntax, I noticed some patterns that are particularly longwinded and counterintuitive. Here are some vague ideas on how to improve the situation:

1. General comparisons

if an operand of a general comparison is a JNode, compare its atomized value(s) with fn:atomic-equal semantics (incomparable types → false), and maps/arrays with fn:deep-equal. This would make the common pattern $in/foo/*[key = true()] correct.

2. Type tests

self::array(*)/array(*) instead of self::jnode(*, array(*)). I believe @ruv has proposed this or something similar before?

3. Flattening

a[].b[] currently becomes (and maybe there’s a shorter variant that I missed):

$in/a/(if (self::jnode(*, array(*))) then * else .)/b/(if (self::jnode(*, array(*))) then * else .)

I think @michaelhkay had the idea before that we could directly jump into arrays (but I believe this was before JNode came up).

4. EBV

Maybe we should compare the ·jvalue· of a JNode when computing the EBV. Otherwise, we have absurd cases like…

let $a := { 'a': () }/a where $a return $a

…where the EBV is true, but the serialized result is empty.

But this may turn out messy, too. An alternative could be a “truthiness” function, to avoid constructs like array:index-of([ (), false(), "", [], {} ], …)

Feedback is welcome, both on the suggestions and the way how I transformed the queries in our test suite.

We state the general principles for the "trusted" and "trust-external" options, but we're not strong on the precise details of what's allowed and what isn't.

• We don't give a precise list of functions that require trust: do they, for example, include system-property(), environment-variable(), and collation-available()?
• We don't say whether partial application or named function references to such functions are allowed
• We say that trustedness is a property of the static context, but how does this relate to the idea that access to a defined whitelist of resources may be permitted? Presumably this is established dynamically. And if trust is a property of the static context, why do we raise a dynamic error?
• There are functions that may or may not access external resources depending on option settings, such as fn:load-xquery-module, fn:transform, and fn:parse-xml. Should we try and say, for example, that a call to parse-xml is allowed provided that it specifies trust-external=false?
• There are things other than functions that require trust, for example import module, import schema, xsl:source-document, xsl:include/import, possibly xsl:result-document. They need to be enumerated.
• Error conditions and codes need to be more precisely defined.

Fix #2989

Changes the rules for load-xquery-module so that values supplied for external global variables are coerced to the required type.

The "Changes" section of XQuery 5.16 says: "The coercion rules are now used when binding values to variables (both global variable declarations and local variable bindings)."

(The term "global variable" has gradually crept in to the spec, which is probably my fault. They are not officially called that.)

This is underpinned by the statement: If a required type is defined, then the value obtained by evaluating the initializing expression is converted to the required type by applying the [coercion rules].

The final paragraph of the section: In all cases the value V must match the type T according to the rules for SequenceType matching; otherwise a [type error] is raised [err:XPTY0004] appears to be an aberration.

In the case of external variables, we say:

if a value is provided for the variable by the external environment, then V is that value. The means by which typed values of external variables are provided by the external environment is implementation-defined.

We don't mention coercion in this case.

For load-xquery-module, we say (for the variables option): The supplied value must conform to the required type of the variable, without conversion. This text is carried over unchanged from 3.1.

It seems unreasonable not to perform coercion in this case. I propose that we should clarify that (a) in the case of external variables, it is implementation-defined what conversions are performed on a supplied value to achieve a value of the required type, and (b) in the specific case of load-xquery-module, coercion is applied.

Fix #2983

Fewer changes than I feared.

For XQUF, we proceed as follows:

• Generated prefixes: on insert and replace, we switch to a new prefix (ns0_1) where XQUF requires XUDY0023.
• Insert: we drpo namespace nodes in the source. XQUF doesn't mention them, so read literally they would be inserted as children.
• Delete: we ignore namespace nodes. XQUF only skips nodes without a parent, so read literally they would be deleted.

Closes #2897

…much better than what we have now, I hope. Feel free to rant at my XSL style; at least it’s properly indented ;)

#2981 wil need to be rebased.

Closes #2984

When doing #2981, I noticed inconsistencies in the presentation of the serialization parameters. For example, allow-duplicate-names is missing from the XSLT table, and the omit-xml-declaration defaults vary (yes/impl-defined/no).

I think we should refactor the spec in this regard and summarize the serialization parameters in one place. I will make an attempt.

The now-merged PR #2966 has introduced inconsistency to the specifications.

The prefatory prose for 6.1.8 invokes the concept of "current position" and that concept is at the heart of the definitions of boundary assertions and all lookahead and lookbehind assertions, but it is no longer part of the definition of metacharacters ^ and $. Either the latter should be redefined in terms of current position, or the other assertions should be redefined without reference to current position, and the initial prose about assertions should be revised to drop any reference to "current position".