"\(" is implementation-defined in C++, and both clang and gcc treat it
as equivalent to "(", not "\\(".
This patch replaces "\(" with "\\(" so that tests with string
interpolation actually use string interpolation.
* bugfix: comments were incorrectly being terminated by CR; for example
jq -n $'1 #foo\r'
fails to compile because the CR character terminates the comment, and
CR is not a valid character in jq syntax.
* improvement: comments fully support Tcl-style line continuation.
Previously this was only "supported" in `-f' scripts, whose first line
starts with "#!", and second line starts with # and ends with \, only
for the comment on the second line, only for one extra line.
* man: document comment syntax, which was previously undocumented.
* tests: add regression tests for the bugfix, and some tests for line
continuation in comments.
This reverts commit 0e70f7a57e.
There is no reason to revert this change.
In #2922, I only disagreed with the commit message suggesting that
LC_CTYPE=C od -t c is equivalent to od -c
The only documented differences are that -tc is required to be
influenced by -N and -j, while -c is not, and that -c is required to
only support a subset of the backslash sequences that -tc should
support.
jq previously only allowed passing the inline script before -- (as if
they were options) even though one would expect the inline script to be
a positional argument.
Since jq previously also refused to run with a usage error if the script
was passed after -- (It was not assuming . as script as it does when
no arguments are passed), and positional arguments are allowed before --
and even before other options, it should not be a breaking change to
change that weird behaviour, and allow the script to appear after --.
It also simplifies the option parsing code a bunch.
Fixes#2918
In process there is a suspicious options |= EXIT_STATUS_EXACT that
is run when the jq script is terminated by halt, or halt_error.
That line of code acutally does nothing because options is a local
argument variable, and is not passed as a pointer. It was probably meant
to be a *options |= EXIT_STATUS_EXACT with the options argument
passed as a int*.
In any case, we do not want to run the code in main() that was supposed
to run if EXIT_STATUS_EXACT is set (but didn't since it is never added
to options); as far as I can tell, we only want to run that code when
the --exit-status/-e option is passed.
So I removed EXIT_STATUS_EXACT completely, and the useless assignment,
instead of fixing it since it was not used for anything else.
Useless assignment detected by clang-tidy.
For the jq_state used by the jq utility, the JQ_LIBRARY_PATH attribute
will always be set, but, in general, it is possible that it might not
be.
If it is not set, jq_get_lib_dirs() will return jv_invalid().
That is not good, because some code in linker.c expects it to always
returns an array.
This patch makes jq_get_lib_dirs() return an empty array if
JQ_LIBRARY_PATH is not set to prevent problems.
This issue made OSS fuzz trigger failed assertions every time it tried
to compile a script that uses "include".
Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=61796
This patch removes the weird behaviour of jv_invalid_with_msg(jv_null())
that returns jv_invalid() (i.e. empty), instead of a boxed jv_null().
The previous behaviour of null|error was obviously unintentional, and
allowing is jv_invalid_with_msg() to return values on which you can't
call jv_invalid_get_msg() is only error prone.
This patch exports all the binary operator builtins functions from
builtin.c and uses them for constant folding in the parser, allowing
constant folding to work will all kinds and combinations of constants.
Now string*number, $ARGS+$ARGS, string/string, etc will also be
constant folded and the implementation of constant folded operators and
runtime operators will be the same.
And thanks to the new ERRORK bytecode operation, errors are constant
folded too! (e.g. 1 / 0 [] * {} etc)
Use a StringStart component that is either FORMAT QQSTRING_START or
QQSTRING_START instead of having two similar rules for String.
This is simpler and avoids having to use an untyped mid-rule action
component to copy FORMAT at the top of the stack before QQString, and
having to use jv_free($<literal>3) instead of jv_free($1) just to make
bison not complain about the "unused" mid-rule component.
You only need to specify the return type in a function pointer
declaration in C.
If you use () in the declaration, the function pointer can be called
with any arguments, and the type of the arguments is decided for each
function call based on the types of the arguments used for the call.
(To declare a function pointer for a function with no arguments, you use
`(void)'.)
Since all the cfunction structs have a fptr that points to a functions
that return jv, not void, we can we can just declare cfunction.fptr as
jv (*)() and avoid having those annoying and technically not C-standard
compliant casts everywhere.
RS should only only have special meaning when parsing json-seq.
Before this fix, RS was sometimes treated as a value terminator, causing
weird results.
Bug discovered by OSS fuzz.
Ref: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=60835
It seems that bison doesn't call destructors for mid-rule action
components on error, since it does not know their type.
A mid-rule action was used to allocate the "text" string used as format
by string literals without a format, which would leak on error.
This patch replaces it with a new NoFormat component of type <literal>.
Now bison will call jv_free() on that string after a syntax error.
Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=60836
$ ./jq '"'
jq: error: syntax error, unexpected end of file, expecting QQSTRING_TEXT or QQSTRING_INTERP_START or QQSTRING_END (Unix shell quoting issues?) at <top-level>, line 1:
"
jq: 1 compile error
=================================================================
==1495450==ERROR: LeakSanitizer: detected memory leaks
Direct leak of 21 byte(s) in 1 object(s) allocated from:
#0 0x7fc21aee1359 in __interceptor_malloc /usr/src/debug/gcc/gcc/libsanitizer/asan/asan_malloc_linux.cpp:69
#1 0x557ccb6ff608 in jv_mem_alloc src/jv_alloc.c:141
SUMMARY: AddressSanitizer: 21 byte(s) leaked in 1 allocation(s).
This is unnecessary since we are using AC_USE_SYSTEM_EXTENSIONS in
configure.ac, which makes the compiler always run with -D_GNU_SOURCE=1
when building for GNU/Linux.
Also, the #define _GNU_SOURCE in src/util.c, and in
src/inject_errors.c (--enable-inject-errors), were not guarded by a
#ifndef _GNU_SOURCE so they were just triggering redefinition warnings.
Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=60691
$ ./jq <<< 0E2202012202
src/decNumber/decNumber.c:598:18: runtime error: signed integer overflow: 440402440 + 1761609760 cannot be represented in type 'int'
0E+999999999
Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=60683
$ ./jq <<< 5E7657765755
src/decNumber/decNumber.c:598:18: runtime error: left shift of 765776575 by 3 places cannot be represented in type 'int'
src/decNumber/decNumber.c:598:18: runtime error: signed integer overflow: 1531553150 + 1831245304 cannot be represented in type 'int'
1.7976931348623157e+308
Only examples defined in the "examples" field are generated, if they are
defined in "example", they are just ignored.
Also fix a bunch of incorrect tests.
Instead of just using {"captures":[]}.
sub functions are use captures for replacement expressions.
If we don't populate, captures for empty matches, the replacement
expression is run with an empty object as input instead of an object
containing the named captures with "" as value:
* before:
$ jq -n '"123foo456bar" | gsub("[^a-z]*(?<x>[a-z]*)"; "Z\(.x)")'
"ZfooZbarZnull"
* after:
$ jq -n '"123foo456bar" | gsub("[^a-z]*(?<x>[a-z]*)"; "Z\(.x)")'
"ZfooZbarZ"
---
I also removed a redundant
result = NULL;
if (result) {
...
}
To make debug/0 not call an uninitialised function pointer when using
--run-tests or when using a jq_state on which jq_set_debug_cb() has not
been called.
To avoid causing segmentation faults when input/1 is called in a
jq_state on which jq_set_input_cb() has not been called; e.g. the one
used by jq --run-tests.
That segfault could also be fixed in run_jq_tests() by calling:
jq_set_input_cb(jq, NULL, NULL);
But I think it makes sense to just make jq_init() initialise those
values to NULL.
Ref: https://github.com/jqlang/jq/pull/2717#discussion_r1264338841
* Fix memory leak for has(nan)
jv_array_get() used to be responsible of freeing the input array, but
since b5c4c3d67d, it is no longer called
if the key is nan. We need to free it manually to avoid leaking the
array.
* Parse nan in JSON as NaN instead of triggering a parse error
Fixes#2021
The markdown python changed its API for markdown extensions.
Bumping the markdown package version in previous commits broke
docs/build_manpage.py since it was using registering an extension with
the old API.
n.b.: { $__loc__: "foo" } is still not allowed.
$__loc__ cannot be overwritten, and it is always an object.
Using { $x: "foo" } where $x is not a string is an error, so there is
not much of a point in supporting that.
Previously, bindings were parsed as the combination of two tokens:
'$' IDENT
This meant that using a keyword as variable name (e.g. $then, $label)
did not work.
Attempts were made to allow $keyword to work by adding some '$' Keyword
rules in the parser, but this did not allow $keyword in all places:
jq --arg label foo -n '$label' # ok
jq -n '"foo" as $label | .' # error
Treating $foo as a single token is much simpler, in my opinion.
This patch also changes how LOC is lexed: "$__loc__" instead of as
"__loc__" that gets combined with '$' in the parser.
This patch also disallows having spaces after '$' when recalling a
variable `$ foo' since that was probably just an unintentional side
effect of the implementation, and it was not documented.
Fixes#2675
Emanuele Torre