QT4 CG Meeting 145 Minutes 2025-12-09

• [X] David J Birnbaum (DB)
• [X] Reece Dunn (RD)
• [X] Sasha Frisov (SF)
• [X] Christian Grün (CG)
• [X] Joel Kalvesmaki (JK) [x:06-]
• [X] Michael Kay (MK)
• [X] Juri Leino (JLO)
• [X] John Lumley (JWL)
• [X] Dimitre Novatchev
• [X] Alan Painter (AP
• [X] Wendell Piez (WP)
• [X] Ruvim Pinka (RP)
• [X] Ed Porter (EP)
• [X] Liam Quin (LQ)
• [X] Norm Tovey-Walsh (NW) Scribe. Chair.

Proposal: Accept the agenda.

Accepted.

Proposal: Accept the minutes of the previous meeting.

Accepted.

The next meeting is planned for 16 December 2025.

Regets: SF, RD

After next week, the CG will recess for the end-of-year holidays. The following meeting will be on 6 January 2026.

• [ ] QT4CG-143-01: CG to make another attempt at binary functions.
• [ ] QT4CG-143-02: MK to try to recover the ability to extract formal equivalences into tests
• [ ] QT4CG-143-03: JK to look for C14N test suites.
• [ ] QT4CG-144-01: MK to consider if any now lost value comparisons should be added as examples.
• [ ] QT4CG-144-02: MK to add notes about edge cases: sequence normalization and character maps for example.

This section summarizes all of the issues and pull requests that need to be resolved before we can finish. See Technical Agenda below for the focus of this meeting.

See PR #2247.

DN begins by reviewing the Deferred Evaluation section of the PR.

• DN: Most useful in cases where it’s impossible or impractical to materialize an entire sequence in memory.
  • … Huge thank you to all the people who supported this work: AP, LQ, Ken Holman, SF, Adam Retter, Andy Bunce, and Kurt Cagle, many of whom expressed enthusiastic support for the work.
• DN: The generator is a way of implementing an iterator.

DN reviews the description of the generator record in the spec.

• NW: Before we get too deep into the details, does LQ have any opening remarks?
• LQ: I sent some notes to the mailing list.
  • … Lots of people want to do this and it’s a bit tricky to write.
  • … If something is a little tricky to write, you’ll get interoperability problems
  • … It’s great to have it readily available.
  • … MK asked about how this relates to XML/XPath/XSLT/XQuery, etc. “There weren’t enough X’s in the proposal.”
  • … It’s sometimes hard to demonstrate some of these things in a small space.
  • … A good comparison is with xsl:iterate in XSLT. Before xsl:iterate, you had to do everything with recursive functions. And that’s tricky if you don’t understand tail-call optimization.
  • … Writing this stuff was quite difficult. It got easier as implementations improved tail-recursion implementations.
  • … The xsl:iterate instruction forces you to write something that can be tail-optimized.
  • … When I asked the XSL mailing list, xsl:iterate was very popular, second only to expand-text.
  • … Generators will be like expand-text; they’ll make lots of things easier to read.
  • … There’s nothing in xsl:iterate that you couldn’t write yourself. You could write fn:replace yourself, too, if you wanted to.
  • … The right question isn’t “what does this let people do that they couldn’t do before”
  • … The right question is “what will they do with it that they wouldn’t have done without it.”

LQ describes his use case (see email).

• LQ: The book lookup case is an example of lazy evaluation. And that’s what generators give us. They let us write in a stereotyped way that would be too expensive if you did all of it, and you know you don’t need to do all of it, but you don’t know how much you need to do.
  • … You can’t download from the British Library, you use an API, and you can’t do them all through that API, that would be a denial of service attack!
• LQ: The business case I have is a text similarity case. It’s expensive to do similarity testing. If you have a 1,000 paragraphs, you might have a million pairs. But you can find candidates with structural information. That reduces it to a few hundred. We want to take the first few and the move on to the next case.
  • … The point is that it’s less to write than a template or function, and it’s in a very common style. Like Javascript’s ?. operator.
• LQ: Things that are good for standardization are things many people have to write themselves, that they tend to write incorrectly, that are just hard enough to do that people will walk away if its not available, and things that extend the reach fo the language.
• RD: One main example of a protoype of this is the random number generator function. Currently in the language, that’s very hard to work with. Having some infrastructure to make working with that easier is a win. Also, this allows you to write other things like a Fibonacci number generator.
• SF: In other languages, generators have been introduced as their own state machine. These coexist with other parts of the system. If you have your own state machine (Fibonacci numbers or decomposing a problem into parts), that makes some problems very easy to write in imperative languages, but much harder in functional languages. Introducing generators opens the ability to solve these sorts of problems.
  • … Iterators, signals, events, etc. It’s an opportunity for coordination across state machines.
  • … These steps can then feed things back.
• SF: I hear MK when he asks about whether it feeds into XSLT. Yes and no, it allows multithreading and multiprocessing to be mixed with procedural programming. That’s not exactly what XSLT does, but it simplifies the code.
• CG: Thanks DN and LQ, for the presentation and the PDF file. I see that you’ve mentioned the while-do function. One reason we introduced this function was to deal with potentially unlimited results. I was wondering if you’ve thought of simple examples that cannot be realized with while-do or are much harder without generators.
• RD: It’s more about feeding this into other things and chaining. The while-do function has the logic in the callback function. But if you wanted to take a list of Fibonacci numbers and in one case you want to multiple them and in another case you want to take the differences, you have to write the Fibonacci logic and tracking in both loops.
• DN: Many of the generator functions are inspired by the while-do function.
• JL: This is really interesting. I’m a big proponent of putting generators in XQuery. I think RD is right that the random number generator is a generator. It would be easier for people to use if they had more infrastructure.
  • … I’d like to know why did you decide to go with a move-next method call instead of something like the yield keyword?
• DN: The short answer is that introducing yield would necessitate changes in the languages that I wanted to avoid. We can do this step by step.
  • … There’s also feedback from C# and other places that suggests that yield makes it harder to understand what is happening.
• MK: Two kinds of comment:
  • … About planning, I’m concerned about the size of the proposal and how much work still needs doing (It has improved a lot! Thank you!)
  • … I look back on past projects of similar size and consider how long they actually took.
  • … In a slightly orthogonal way, it’s great to see the number of people we have active in this group. A project manager once told me there are good starters and good finishers. And in this kind of activity, we have to worry about good finishers. I’m concerned about that in terms of working group dynamics.
  • … The other concern I have is a sort of feeling that it could be much smaller. I’d be able to answer that more clearly, if I had three or four worked out use cases using generators laid out in front of me. Then I could compare. That’s an appeal to do more work on use cases.
• AP: I wanted to second what MK said about the use cases. I think that would be very useful. DN and I were working on the slice function and there was a bit of a missmatch there.
  • … I also have a gut feeling that things could be simplified a bit. Maybe it could be a library module? Maybe we could have a separate definition for generators.
• JWL: Very similar. I got three senses: the generator, the “necessary” or “most preferable” functions, and then you’ve got a lot of other functions that sit on top of it. What is the essense that needs to be implemented deep down to get performance or the right kinds of outputs.
• JK: I support the idea of generators. I feel the pain of implementors. As a point of suggestion on how to bridge this gap, I recommend the authors take a more real world example with an infinite amount of XML. I think that would help.
• NW: Are you guys willing to provide three or four substantial worked up use cases?

LQ and DN seem generally willing.

• CG: I would also be interested in exploring the exising XQuery implementation and if there are core features that would make it run more quickly, I’d be interested in that.
• DN: There’s a pure XQuery implementation: https://github.com/dnovatchev/Articles/blob/main/Generators/Code/generator.xq
  • … I think trying that out would be very useful. It might be possible for people to make their own use cases.

We’ll return to this PR in a few weeks when use cases have been produced and reviewed.