When the string transformation to number failed, all following
transformation failed too.
This happend because status in decNumberFromString function is
updated just in error case. Reusing the DEC_CONTEXT that failed
before results into error even if the string is valid number.
We need to be careful to not grab an extra reference when mutating data
structures because that means we make extra copies. Doing that every
time in `_modify` is really painful, as that function implements `|=`
and all modify-assign operators.
`jv_setpath()` also grabs additional references, so this is not the only
fix needed for the modify-assign operators to not be accidentally
quadratic.
We have to use `LOADVN` in order to make this fix possible, so we should
really byte-code `_modify` rather than jq-code it. However, as a
stop-gap to make this fix easier, I'm adding syntax for referring to a
`$binding` such that you get `LOADVN` instead of `LOADV`.
This syntax is not meant to be used outside jq's internals! Therefore
it is not documented. I promise to break it later, so don't use it!
TBD, but later:
- Optimize `_modify` for the case where `update` outputs more than one
value.
This is trivial: add a `setpath($p; null)` in the middle of `_modify`
before calling `update`. But unfortunately the VM retains a
reference to `value_at_path` for path expression checking, and fixing
that will require more work.
We need to be careful to not grab an extra reference when mutating
data structures because that means we make extra copies. Doing that
every time in `jv_setpath()` is really painful, as that function is used
in `_modify/2`, which is used in `|=` and all modify-assign operators.
`_modify` also grabs additional references, so this is not the only fix
needed for the modify-assign operators to not be accidentally quadratic.
It has no effect after the change c4524da.
Bug reported by the clang static analyzer.
Description: Value stored to 'value' is never read
File: jq/src/util.c
Line: 439
Allow building jq in a mode that doesn't use decnumber for benchmarking
purposes. decnumber support is enabled by default, and this option is
meant to be removed once we're happy with the performance.
Extend jv_number to use decNumber for storing number literals. Any math
operations on the numbers will truncate them to double precision.
Comparisons when both numbers are literal numbers will compare them
without truncation.
Delay conversion of numbers to doubles until a math operation is performed,
to preserve precision. A literal jv_number will only need conversion to
double once, and will reuse the resultant double on subsequent
conversions.
Outputting literal jv_numbers preserves the original precision.
Add strong pthread requirement to manage contexts/allocations for
converting numbers between their decNumber, string, and double formats.
Expressions of the form `path(EXPR) | select(GENERATOR)`, where `EXPR`
is a path expression and `GENERATOR` is a generator conditional
expression (e.g., `has("a"), has("b")`) cause an assertion if the
jq_state VM is torn down too soon. That assert() was only correct if
assuming that the conditional is not a generator.
If the conditional is generator, then what we see is that when
backtracking a SUBEXP_END is executed without a corresponding
SUBEXP_BEGIN because the entire conditional is bracketed with
SUBEXP_BEGIN and SUBEXP_END, and since it's resumed in the middle, in
between the brackets.
Rather than assert that the jq->path_len being restored has some
particular value, we can simply re-compute it from the restored
jq->path.
The behavior of memmem for an empty needle is inconsistent between
implementations of libc.
Our tests imply that we want an empty string needle to be true,
so check for an empty needle before calling memmem.
Owen Ou