Improve performance of join/1 on large arrays (#3639)

join/1 folded each element as `(acc + $x) + elem`, with the element
branch ending in `.//""`. Both leave a second live reference to the
accumulator: the nested + feeds the same `.` to both operands, and the
`//` leaves a forkpoint below it on the stack. `jvp_string_append`
therefore cannot grow the accumulator in place, and copies the whole
string once per element.

Pipe `acc + $x` into `. + elem` and drop the `//`, since adding null to
a string is already a no-op. join/1 is now linear: joining 8000
strings of 1000 chars goes from 6.5s to 0.06s, with output and error
messages unchanged.
This commit is contained in:
Artem Lytkin authored and GitHub committed 2026-09-27 13:52:41 +09:00
1 parent 32580f5d83
commit 9a75bb0d43
2 files changed
+12 -2

No files matched your search

+4 -2
View File
@@ -63,9 +63,11 @@ def strings: select(type == "string");
def nulls: select(. == null);
def values: select(. != null);
def scalars: select(type|. != "array" and . != "object");
# Keep the accumulator the only reference on the left of + so the string is
# appended in place; a nested + or a // inside the reduce copies it every step.
def join($x): reduce .[] as $i (null;
(if .==null then "" else .+$x end) +
($i | if type=="boolean" or type=="number" then tostring else .//"" end)
(if .==null then "" else .+$x end) |
. + ($i | if type=="boolean" or type=="number" then tostring end)
) // "";
def _flatten($x): reduce .[] as $i ([]; if $i | type == "array" and $x != 0 then . + ($i | _flatten($x-1)) else . + [$i] end);
def flatten($x): if $x < 0 then error("flatten depth must not be negative") else _flatten($x) end;