Parse the full ISO 8601 grammar in fromdateiso8601 (#3617)
`fromdateiso8601` was `strptime("%Y-%m-%dT%H:%M:%SZ") | mktime`, which
accepts only one of the many representations ISO 8601 permits. strptime
cannot be extended to cover the rest portably, since it has no specifier
for fractional seconds and glibc discards `%G` and `%V`.
Replace it with a parser in src/iso8601.c that handles basic and extended
representations, ordinal and week dates, fractional seconds, offsets and
date-only input. It uses integer arithmetic instead of `time_t`, so the
results no longer depend on the platform.
This is stricter than before. Impossible dates like `2015-02-30`, which
`mktime` used to normalize into March, and unpadded components like
`2015-3-5T6:00:00Z` are now errors. So is any trailing input, including
whitespace. Leading whitespace is rejected too; before, glibc skipped it
and macOS did not. A leap second now maps to the second before it rather
than the next minute. The error message has changed, and the result can
now be non-integer.
This commit is contained in:
itchyny authored and GitHub committed
1 parent
a19efdd45d
commit
fd25c3e720
10 files changed
+560
-20
No files matched your search
+3
-2
@@ -5,7 +5,8 @@ LIBJQ_INCS = src/builtin.h src/bytecode.h src/compile.h \
|
||||
src/exec_stack.h src/jq_parser.h src/jv_alloc.h src/jv_dtoa.h \
|
||||
src/jv_unicode.h src/jv_utf8_tables.h src/lexer.l src/libm.h \
|
||||
src/linker.h src/locfile.h src/opcode_list.h src/parser.y \
|
||||
src/util.h src/jv_dtoa_tsd.h src/jv_thread.h src/jv_private.h \
|
||||
src/util.h src/iso8601.h \
|
||||
src/jv_dtoa_tsd.h src/jv_thread.h src/jv_private.h \
|
||||
vendor/decNumber/decContext.h vendor/decNumber/decNumber.h \
|
||||
vendor/decNumber/decNumberLocal.h
|
||||
|
||||
@@ -13,7 +14,7 @@ LIBJQ_SRC = src/builtin.c src/bytecode.c src/compile.c src/execute.c \
|
||||
src/jq_test.c src/jv.c src/jv_alloc.c src/jv_aux.c \
|
||||
src/jv_dtoa.c src/jv_file.c src/jv_parse.c src/jv_print.c \
|
||||
src/jv_unicode.c src/linker.c src/locfile.c src/util.c \
|
||||
src/jv_dtoa_tsd.c \
|
||||
src/iso8601.c src/jv_dtoa_tsd.c \
|
||||
vendor/decNumber/decContext.c vendor/decNumber/decNumber.c \
|
||||
${LIBJQ_INCS}
|
||||
|
||||
|
||||
@@ -2250,13 +2250,45 @@ sections:
|
||||
|
||||
The `fromdateiso8601` builtin parses datetimes in the ISO 8601
|
||||
format to a number of seconds since the Unix epoch
|
||||
(1970-01-01T00:00:00Z). The `todateiso8601` builtin does the
|
||||
inverse.
|
||||
(1970-01-01T00:00:00Z). The `todateiso8601` builtin formats a
|
||||
number of seconds as `"%Y-%m-%dT%H:%M:%SZ"`.
|
||||
|
||||
The `fromdate` builtin parses datetime strings. Currently
|
||||
`fromdate` only supports ISO 8601 datetime strings, but in the
|
||||
future it will attempt to parse datetime strings in more
|
||||
formats.
|
||||
`fromdateiso8601` accepts a complete date, optionally followed by
|
||||
a time of day, written in either the extended representation, which
|
||||
separates the components, or the basic representation, which does
|
||||
not. The two cannot be mixed within one datetime. The date may be
|
||||
a calendar date (`2015-03-05`, `20150305`), an ordinal date
|
||||
(`2015-064`, `2015064`), or a week date (`2015-W10-4`, `2015W104`);
|
||||
all three denote the same day. Years outside the range 0000 to 9999
|
||||
are written with a sign and six digits, as in `+012015-03-05`.
|
||||
|
||||
The time of day may omit its seconds, or its minutes and seconds,
|
||||
as in `2015-03-05T23:51Z` and `2015-03-05T23Z`, or be left out
|
||||
altogether, in which case the value is midnight UTC. It may carry a
|
||||
fractional part after a `.` or a `,`, which belongs to the lowest
|
||||
component present, so `23,5` is half past 23. It may be followed
|
||||
by an offset from UTC written as `Z`, `+hh`, `+hh:mm` or `+hhmm`;
|
||||
a datetime with no offset is read as UTC. A time of `24:00:00`
|
||||
denotes midnight of the following day, and a leap second is read as
|
||||
the second before it, since jq counts every day as 86400 seconds.
|
||||
|
||||
The `W`, `T` and `Z` designators may also be written in lower
|
||||
case, which ISO 8601 permits where upper case is not available.
|
||||
RFC 3339 additionally allows the date and the time to be separated
|
||||
by a space instead of a `T` "for the sake of readability", which jq
|
||||
accepts too. The offset may use the other representation's
|
||||
separator style, as in `2015-03-05T23:51:47+0900`, although
|
||||
ISO 8601 does not allow the basic and extended representations
|
||||
to be mixed.
|
||||
|
||||
A complete date is required, so `2015` and `2015-03` are errors
|
||||
even though ISO 8601 defines them as dates, and `T23:51:47` is an
|
||||
error because a time of day on its own is not a point in time.
|
||||
Dates that do not exist, such as `2015-02-30` or `2016-W53-1`, are
|
||||
errors too, as is `2015-03-05Z`, since a zone designator belongs to
|
||||
the time of day.
|
||||
|
||||
The `fromdate` builtin is an alias for `fromdateiso8601`.
|
||||
|
||||
The `todate` builtin is an alias for `todateiso8601`.
|
||||
|
||||
@@ -2296,8 +2328,11 @@ sections:
|
||||
the local timezone setting.
|
||||
|
||||
The format strings for `strptime` and `strftime` are described
|
||||
in typical C library documentation. The format string for ISO
|
||||
8601 datetime is `"%Y-%m-%dT%H:%M:%SZ"`.
|
||||
in typical C library documentation. `"%Y-%m-%dT%H:%M:%SZ"` is
|
||||
the format that `todateiso8601` produces; it is only one of the
|
||||
ISO 8601 representations that `fromdateiso8601` accepts, and
|
||||
`strptime` cannot parse the others -- there is no portable format
|
||||
specifier for week dates or for fractional seconds.
|
||||
|
||||
jq may not support some or all of this date functionality on
|
||||
some systems. In particular, the `%u` and `%j` specifiers for
|
||||
@@ -2308,6 +2343,14 @@ sections:
|
||||
input: '"2015-03-05T23:51:47Z"'
|
||||
output: ['1425599507']
|
||||
|
||||
- program: 'fromdateiso8601'
|
||||
input: '"2015-03-06T08:51:47.25+09:00"'
|
||||
output: ['1425599507.25']
|
||||
|
||||
- program: 'map(fromdateiso8601)'
|
||||
input: '["20150305T235147Z","2015-064T23:51:47Z","2015-W10-4T23:51:47Z"]'
|
||||
output: ['[1425599507,1425599507,1425599507]']
|
||||
|
||||
- program: 'strptime("%Y-%m-%dT%H:%M:%SZ")'
|
||||
input: '"2015-03-05T23:51:47Z"'
|
||||
output: ['[2015,2,5,23,51,47,4,63]']
|
||||
|
||||
Generated
+24
-4
@@ -1,5 +1,5 @@
|
||||
.
|
||||
.TH "JQ" "1" "May 2025" "" ""
|
||||
.TH "JQ" "1" "August 2026" "" ""
|
||||
.
|
||||
.SH "NAME"
|
||||
\fBjq\fR \- Command\-line JSON processor
|
||||
@@ -2457,10 +2457,22 @@ jq \'@base64d\'
|
||||
jq provides some basic date handling functionality, with some high\-level and low\-level builtins\. In all cases these builtins deal exclusively with time in UTC\.
|
||||
.
|
||||
.P
|
||||
The \fBfromdateiso8601\fR builtin parses datetimes in the ISO 8601 format to a number of seconds since the Unix epoch (1970\-01\-01T00:00:00Z)\. The \fBtodateiso8601\fR builtin does the inverse\.
|
||||
The \fBfromdateiso8601\fR builtin parses datetimes in the ISO 8601 format to a number of seconds since the Unix epoch (1970\-01\-01T00:00:00Z)\. The \fBtodateiso8601\fR builtin formats a number of seconds as \fB"%Y\-%m\-%dT%H:%M:%SZ"\fR\.
|
||||
.
|
||||
.P
|
||||
The \fBfromdate\fR builtin parses datetime strings\. Currently \fBfromdate\fR only supports ISO 8601 datetime strings, but in the future it will attempt to parse datetime strings in more formats\.
|
||||
\fBfromdateiso8601\fR accepts a complete date, optionally followed by a time of day, written in either the extended representation, which separates the components, or the basic representation, which does not\. The two cannot be mixed within one datetime\. The date may be a calendar date (\fB2015\-03\-05\fR, \fB20150305\fR), an ordinal date (\fB2015\-064\fR, \fB2015064\fR), or a week date (\fB2015\-W10\-4\fR, \fB2015W104\fR); all three denote the same day\. Years outside the range 0000 to 9999 are written with a sign and six digits, as in \fB+012015\-03\-05\fR\.
|
||||
.
|
||||
.P
|
||||
The time of day may omit its seconds, or its minutes and seconds, as in \fB2015\-03\-05T23:51Z\fR and \fB2015\-03\-05T23Z\fR, or be left out altogether, in which case the value is midnight UTC\. It may carry a fractional part after a \fB\.\fR or a \fB,\fR, which belongs to the lowest component present, so \fB23,5\fR is half past 23\. It may be followed by an offset from UTC written as \fBZ\fR, \fB+hh\fR, \fB+hh:mm\fR or \fB+hhmm\fR; a datetime with no offset is read as UTC\. A time of \fB24:00:00\fR denotes midnight of the following day, and a leap second is read as the second before it, since jq counts every day as 86400 seconds\.
|
||||
.
|
||||
.P
|
||||
The \fBW\fR, \fBT\fR and \fBZ\fR designators may also be written in lower case, which ISO 8601 permits where upper case is not available\. RFC 3339 additionally allows the date and the time to be separated by a space instead of a \fBT\fR "for the sake of readability", which jq accepts too\. The offset may use the other representation\'s separator style, as in \fB2015\-03\-05T23:51:47+0900\fR, although ISO 8601 does not allow the basic and extended representations to be mixed\.
|
||||
.
|
||||
.P
|
||||
A complete date is required, so \fB2015\fR and \fB2015\-03\fR are errors even though ISO 8601 defines them as dates, and \fBT23:51:47\fR is an error because a time of day on its own is not a point in time\. Dates that do not exist, such as \fB2015\-02\-30\fR or \fB2016\-W53\-1\fR, are errors too, as is \fB2015\-03\-05Z\fR, since a zone designator belongs to the time of day\.
|
||||
.
|
||||
.P
|
||||
The \fBfromdate\fR builtin is an alias for \fBfromdateiso8601\fR\.
|
||||
.
|
||||
.P
|
||||
The \fBtodate\fR builtin is an alias for \fBtodateiso8601\fR\.
|
||||
@@ -2487,7 +2499,7 @@ The \fBstrptime(fmt)\fR builtin parses input strings matching the \fBfmt\fR argu
|
||||
The \fBstrftime(fmt)\fR builtin formats a time (GMT) with the given format\. The \fBstrflocaltime\fR does the same, but using the local timezone setting\.
|
||||
.
|
||||
.P
|
||||
The format strings for \fBstrptime\fR and \fBstrftime\fR are described in typical C library documentation\. The format string for ISO 8601 datetime is \fB"%Y\-%m\-%dT%H:%M:%SZ"\fR\.
|
||||
The format strings for \fBstrptime\fR and \fBstrftime\fR are described in typical C library documentation\. \fB"%Y\-%m\-%dT%H:%M:%SZ"\fR is the format that \fBtodateiso8601\fR produces; it is only one of the ISO 8601 representations that \fBfromdateiso8601\fR accepts, and \fBstrptime\fR cannot parse the others \-\- there is no portable format specifier for week dates or for fractional seconds\.
|
||||
.
|
||||
.P
|
||||
jq may not support some or all of this date functionality on some systems\. In particular, the \fB%u\fR and \fB%j\fR specifiers for \fBstrptime(fmt)\fR are not supported on macOS\.
|
||||
@@ -2500,6 +2512,14 @@ jq \'fromdate\'
|
||||
"2015\-03\-05T23:51:47Z"
|
||||
=> 1425599507
|
||||
|
||||
jq \'fromdateiso8601\'
|
||||
"2015\-03\-06T08:51:47\.25+09:00"
|
||||
=> 1425599507\.25
|
||||
|
||||
jq \'map(fromdateiso8601)\'
|
||||
["20150305T235147Z","2015\-064T23:51:47Z","2015\-W10\-4T23:51:47Z"]
|
||||
=> [1425599507,1425599507,1425599507]
|
||||
|
||||
jq \'strptime("%Y\-%m\-%dT%H:%M:%SZ")\'
|
||||
"2015\-03\-05T23:51:47Z"
|
||||
=> [2015,2,5,23,51,47,4,63]
|
||||
|
||||
@@ -32,6 +32,7 @@
|
||||
#include "bytecode.h"
|
||||
#include "linker.h"
|
||||
#include "locfile.h"
|
||||
#include "iso8601.h"
|
||||
#include "jv_unicode.h"
|
||||
#include "jv_alloc.h"
|
||||
#include "jv_dtoa.h"
|
||||
@@ -1680,6 +1681,17 @@ static jv f_strptime(jq_state *jq, jv a, jv b) {
|
||||
return r;
|
||||
}
|
||||
|
||||
static jv f_fromdateiso8601(jq_state *jq, jv a) {
|
||||
if (jv_get_kind(a) != JV_KIND_STRING)
|
||||
return ret_error(a, jv_string("fromdateiso8601 requires string inputs"));
|
||||
double t;
|
||||
if (!iso8601_parse(jv_string_value(a), jv_string_length_bytes(jv_copy(a)), &t))
|
||||
return ret_error(a, jv_string_fmt("date \"%s\" is not a valid ISO 8601 datetime",
|
||||
jv_string_value(a)));
|
||||
jv_free(a);
|
||||
return jv_number(t);
|
||||
}
|
||||
|
||||
static int jv2tm(jv a, struct tm *tm, int localtime) {
|
||||
memset(tm, 0, sizeof(*tm));
|
||||
static const size_t offsets[] = {
|
||||
@@ -2041,6 +2053,7 @@ BINOPS
|
||||
CFUNC(f_input, "input", 1),
|
||||
CFUNC(f_debug, "debug", 1),
|
||||
CFUNC(f_stderr, "stderr", 1),
|
||||
CFUNC(f_fromdateiso8601, "fromdateiso8601", 1),
|
||||
CFUNC(f_strptime, "strptime", 2),
|
||||
CFUNC(f_strftime, "strftime", 2),
|
||||
CFUNC(f_strflocaltime, "strflocaltime", 2),
|
||||
|
||||
@@ -74,7 +74,6 @@ def flatten($x): if $x < 0 then error("flatten depth must not be negative") else
|
||||
def flatten: _flatten(-1);
|
||||
def range($x;$y): range($x;$y;1);
|
||||
def range($x): range(0;$x);
|
||||
def fromdateiso8601: strptime("%Y-%m-%dT%H:%M:%SZ")|mktime;
|
||||
def todateiso8601: strftime("%Y-%m-%dT%H:%M:%SZ");
|
||||
def fromdate: fromdateiso8601;
|
||||
def todate: todateiso8601;
|
||||
|
||||
+342
@@ -0,0 +1,342 @@
|
||||
/*
|
||||
* A self-contained ISO 8601 date and time parser.
|
||||
*
|
||||
* The C library is of no help here: strptime() has no format specifier for
|
||||
* fractional seconds, and no portable way to parse week dates (glibc
|
||||
* consumes %G and %V but then discards them). The results would also
|
||||
* differ between the host strptime() and the copy bundled in util.c.
|
||||
* Everything below is plain integer arithmetic instead, which also keeps
|
||||
* the computation clear of the range of a 32-bit time_t.
|
||||
*
|
||||
* The accepted grammar is
|
||||
*
|
||||
* datetime := date [ ("T"|"t"|" ") time ]
|
||||
* date := year "-" MM "-" DD | year MMDD
|
||||
* | year "-" DDD | year DDD
|
||||
* | year "-" ("W"|"w") ww "-" d | year ("W"|"w") ww d
|
||||
* year := YYYY | ("+"|"-") YYYYYY
|
||||
* time := hms [ ("."|",") digits ] [ offset ]
|
||||
* hms := hh [ ":" mm [ ":" ss ] ] | hh [ mm [ ss ] ]
|
||||
* offset := ("Z"|"z") | ("+"|"-") hh [ [":"] mm ]
|
||||
*
|
||||
* As in the standard's own notation a run of letters stands for that many
|
||||
* decimal digits -- DDD is the three-digit day of the year, d the one-digit
|
||||
* day of the week -- while "digits" is one or more of them.
|
||||
*
|
||||
* The decimal fraction belongs to the lowest component of the hms that is
|
||||
* present, so "23,5" is half past 23 and "2320,8" is 23:20:48.
|
||||
*
|
||||
* The date must always be complete, so "2015-03" is an error even though it
|
||||
* is a valid ISO 8601 date. The time may drop its seconds, or its minutes
|
||||
* and seconds, or be omitted altogether, in which case the value is
|
||||
* midnight UTC. The date and the time must agree on whether separators are
|
||||
* used, as ISO 8601 requires -- "2026-08-28T093015Z" is not a valid
|
||||
* representation. The offset is exempt from that rule, since "+0900" after
|
||||
* an extended time is common in the wild.
|
||||
*/
|
||||
|
||||
#include <stdint.h>
|
||||
|
||||
#include "iso8601.h"
|
||||
|
||||
/*
|
||||
* Days since 1970-01-01 of the civil date y-m-d in the proleptic Gregorian
|
||||
* calendar, negative before the epoch, after Howard Hinnant's
|
||||
* days_from_civil(). Requires 1 <= m <= 12 and 1 <= d <= days_in_month(y, m);
|
||||
* the callers check both.
|
||||
*/
|
||||
static int64_t days_from_civil(int64_t y, int m, int d) {
|
||||
y -= m <= 2;
|
||||
int64_t era = (y >= 0 ? y : y - 399) / 400;
|
||||
int64_t yoe = y - era * 400; /* [0, 399] */
|
||||
int64_t doy = (153 * (m + (m > 2 ? -3 : 9)) + 2) / 5 + d - 1; /* [0, 365] */
|
||||
int64_t doe = yoe * 365 + yoe / 4 - yoe / 100 + doy; /* [0, 146096] */
|
||||
return era * 146097 + doe - 719468;
|
||||
}
|
||||
|
||||
static int is_leap(int64_t y) {
|
||||
return (y % 4 == 0 && y % 100 != 0) || y % 400 == 0;
|
||||
}
|
||||
|
||||
static int days_in_month(int64_t y, int m) {
|
||||
static const int d[] = {31, 28, 31, 30, 31, 30, 31, 31, 30, 31, 30, 31};
|
||||
return m == 2 && is_leap(y) ? 29 : d[m - 1];
|
||||
}
|
||||
|
||||
/* ISO weekday (1 = Monday .. 7 = Sunday) of a day count since the epoch. */
|
||||
static int iso_wday(int64_t days) {
|
||||
int64_t w = (days + 3) % 7;
|
||||
if (w < 0)
|
||||
w += 7;
|
||||
return (int)w + 1;
|
||||
}
|
||||
|
||||
/*
|
||||
* An ISO year has 53 weeks if it starts on a Thursday, or if it is a leap
|
||||
* year starting on a Wednesday; otherwise it has 52.
|
||||
*/
|
||||
static int iso_weeks_in_year(int64_t y) {
|
||||
int jan1 = iso_wday(days_from_civil(y, 1, 1));
|
||||
return jan1 == 4 || (jan1 == 3 && is_leap(y)) ? 53 : 52;
|
||||
}
|
||||
|
||||
static int civil_to_days(int64_t y, int mon, int mday, int64_t *days) {
|
||||
if (mon < 1 || mon > 12 || mday < 1 || mday > days_in_month(y, mon))
|
||||
return 0;
|
||||
*days = days_from_civil(y, mon, mday);
|
||||
return 1;
|
||||
}
|
||||
|
||||
static int ordinal_to_days(int64_t y, int yday, int64_t *days) {
|
||||
if (yday < 1 || yday > (is_leap(y) ? 366 : 365))
|
||||
return 0;
|
||||
*days = days_from_civil(y, 1, 1) + yday - 1;
|
||||
return 1;
|
||||
}
|
||||
|
||||
static int week_to_days(int64_t y, int week, int wday, int64_t *days) {
|
||||
if (week < 1 || week > iso_weeks_in_year(y) || wday < 1 || wday > 7)
|
||||
return 0;
|
||||
int64_t jan4 = days_from_civil(y, 1, 4);
|
||||
*days = jan4 - (iso_wday(jan4) - 1) + (int64_t)(week - 1) * 7 + (wday - 1);
|
||||
return 1;
|
||||
}
|
||||
|
||||
/*
|
||||
* The scanner works over an explicit [p, end) range rather than a C string, so
|
||||
* that a NUL byte embedded in the input is a parse error instead of silently
|
||||
* truncating the value.
|
||||
*/
|
||||
typedef struct {
|
||||
const char *p;
|
||||
const char *end;
|
||||
} iso8601_scanner;
|
||||
|
||||
static int scan_at_end(iso8601_scanner *s) {
|
||||
return s->p == s->end;
|
||||
}
|
||||
|
||||
static int scan_peek(iso8601_scanner *s) {
|
||||
return scan_at_end(s) ? -1 : (unsigned char)*s->p;
|
||||
}
|
||||
|
||||
static int scan_eat(iso8601_scanner *s, char c) {
|
||||
if (scan_at_end(s) || *s->p != c)
|
||||
return 0;
|
||||
s->p++;
|
||||
return 1;
|
||||
}
|
||||
|
||||
/*
|
||||
* W, T and Z are what ISO 8601 calls designators. They are written in upper
|
||||
* case, but the standard notes that "lower case characters may be used when
|
||||
* upper case characters are not available", so match either.
|
||||
*/
|
||||
static int scan_designator(iso8601_scanner *s, char c) {
|
||||
if (scan_at_end(s) || (*s->p != c && *s->p != c + ('a' - 'A')))
|
||||
return 0;
|
||||
s->p++;
|
||||
return 1;
|
||||
}
|
||||
|
||||
static size_t scan_count_digits(iso8601_scanner *s) {
|
||||
size_t n = 0;
|
||||
while (s->p + n < s->end && s->p[n] >= '0' && s->p[n] <= '9')
|
||||
n++;
|
||||
return n;
|
||||
}
|
||||
|
||||
static int scan_digits(iso8601_scanner *s, int n, int *out) {
|
||||
if (s->end - s->p < n)
|
||||
return 0;
|
||||
int v = 0;
|
||||
for (int i = 0; i < n; i++) {
|
||||
unsigned char c = (unsigned char)s->p[i];
|
||||
if (c < '0' || c > '9')
|
||||
return 0;
|
||||
v = v * 10 + (c - '0');
|
||||
}
|
||||
s->p += n;
|
||||
*out = v;
|
||||
return 1;
|
||||
}
|
||||
|
||||
/*
|
||||
* A plain year is exactly four digits. An expanded year carries a mandatory
|
||||
* sign and exactly six digits, which is what makes it unambiguous in front of
|
||||
* a date in the basic form.
|
||||
*/
|
||||
static int parse_year(iso8601_scanner *s, int64_t *year) {
|
||||
int64_t sign = 1;
|
||||
int digits = 4;
|
||||
if (scan_peek(s) == '+' || scan_peek(s) == '-') {
|
||||
sign = *s->p == '-' ? -1 : 1;
|
||||
s->p++;
|
||||
digits = 6;
|
||||
}
|
||||
if (s->end - s->p < digits)
|
||||
return 0;
|
||||
int64_t v = 0;
|
||||
for (int i = 0; i < digits; i++) {
|
||||
unsigned char c = (unsigned char)s->p[i];
|
||||
if (c < '0' || c > '9')
|
||||
return 0;
|
||||
v = v * 10 + (c - '0');
|
||||
}
|
||||
s->p += digits;
|
||||
*year = sign * v;
|
||||
return 1;
|
||||
}
|
||||
|
||||
/* Sets *extended to whether the date was written with "-" separators. */
|
||||
static int parse_date(iso8601_scanner *s, int64_t *days, int *extended) {
|
||||
int64_t year;
|
||||
int week, wday, mon, mday, yday;
|
||||
|
||||
if (!parse_year(s, &year))
|
||||
return 0;
|
||||
|
||||
if (scan_eat(s, '-')) {
|
||||
*extended = 1;
|
||||
if (scan_designator(s, 'W')) {
|
||||
return scan_digits(s, 2, &week) && scan_eat(s, '-') &&
|
||||
scan_digits(s, 1, &wday) && week_to_days(year, week, wday, days);
|
||||
}
|
||||
switch (scan_count_digits(s)) {
|
||||
case 3:
|
||||
return scan_digits(s, 3, &yday) && ordinal_to_days(year, yday, days);
|
||||
case 2:
|
||||
return scan_digits(s, 2, &mon) && scan_eat(s, '-') &&
|
||||
scan_digits(s, 2, &mday) && civil_to_days(year, mon, mday, days);
|
||||
default:
|
||||
return 0;
|
||||
}
|
||||
}
|
||||
|
||||
*extended = 0;
|
||||
if (scan_designator(s, 'W')) {
|
||||
return scan_digits(s, 2, &week) && scan_digits(s, 1, &wday) &&
|
||||
week_to_days(year, week, wday, days);
|
||||
}
|
||||
switch (scan_count_digits(s)) {
|
||||
case 4:
|
||||
return scan_digits(s, 2, &mon) && scan_digits(s, 2, &mday) &&
|
||||
civil_to_days(year, mon, mday, days);
|
||||
case 3:
|
||||
return scan_digits(s, 3, &yday) && ordinal_to_days(year, yday, days);
|
||||
default:
|
||||
/* A bare "YYYYMM" is ambiguous with "YYMMDD"; the standard forbids it. */
|
||||
return 0;
|
||||
}
|
||||
}
|
||||
|
||||
/*
|
||||
* Parses the time of day, storing the seconds since midnight in *secs. A
|
||||
* time of "24:00:00" is midnight of the following day, which *carry reports.
|
||||
*
|
||||
* The minutes and seconds may be omitted, in which case the decimal fraction
|
||||
* belongs to the lowest component that is present -- "23,5" is half past 23
|
||||
* -- so *unit* records how many seconds that component is worth.
|
||||
*/
|
||||
static int parse_time(iso8601_scanner *s, int extended, double *secs,
|
||||
int64_t *carry) {
|
||||
int hh, mm = 0, ss = 0, unit = 3600;
|
||||
|
||||
if (!scan_digits(s, 2, &hh))
|
||||
return 0;
|
||||
if (extended ? scan_eat(s, ':') : scan_count_digits(s) >= 2) {
|
||||
if (!scan_digits(s, 2, &mm))
|
||||
return 0;
|
||||
unit = 60;
|
||||
if (extended ? scan_eat(s, ':') : scan_count_digits(s) >= 2) {
|
||||
if (!scan_digits(s, 2, &ss))
|
||||
return 0;
|
||||
unit = 1;
|
||||
}
|
||||
}
|
||||
|
||||
double frac = 0;
|
||||
if (scan_peek(s) == '.' || scan_peek(s) == ',') {
|
||||
static const double pow10[] = {
|
||||
1e0, 1e1, 1e2, 1e3, 1e4, 1e5, 1e6, 1e7,
|
||||
1e8, 1e9, 1e10, 1e11, 1e12, 1e13, 1e14, 1e15
|
||||
};
|
||||
s->p++;
|
||||
if (scan_count_digits(s) == 0)
|
||||
return 0;
|
||||
uint64_t num = 0;
|
||||
int digits = 0;
|
||||
while (scan_peek(s) >= '0' && scan_peek(s) <= '9') {
|
||||
if (digits < 15) {
|
||||
num = num * 10 + (uint64_t)(*s->p - '0');
|
||||
digits++;
|
||||
}
|
||||
s->p++;
|
||||
}
|
||||
frac = (double)num / pow10[digits];
|
||||
}
|
||||
|
||||
*carry = 0;
|
||||
if (hh == 24) {
|
||||
if (mm != 0 || ss != 0 || frac != 0)
|
||||
return 0;
|
||||
hh = 0;
|
||||
*carry = 1;
|
||||
} else if (hh > 24 || mm > 59 || ss > 60) {
|
||||
return 0;
|
||||
} else if (ss == 60) {
|
||||
ss = 59;
|
||||
}
|
||||
|
||||
*secs = hh * 3600 + mm * 60 + ss + frac * unit;
|
||||
return 1;
|
||||
}
|
||||
|
||||
/* An absent offset means UTC, which is what jq has always assumed. */
|
||||
static int parse_offset(iso8601_scanner *s, int *offset) {
|
||||
if (scan_designator(s, 'Z')) {
|
||||
*offset = 0;
|
||||
return 1;
|
||||
}
|
||||
|
||||
int sign = scan_peek(s);
|
||||
if (sign != '+' && sign != '-') {
|
||||
*offset = 0;
|
||||
return 1;
|
||||
}
|
||||
|
||||
s->p++;
|
||||
int oh, om = 0;
|
||||
if (!scan_digits(s, 2, &oh))
|
||||
return 0;
|
||||
int had_colon = scan_eat(s, ':');
|
||||
if (!scan_digits(s, 2, &om) && had_colon)
|
||||
return 0;
|
||||
if (oh > 23 || om > 59)
|
||||
return 0;
|
||||
|
||||
*offset = (sign == '+' ? 1 : -1) * (oh * 3600 + om * 60);
|
||||
return 1;
|
||||
}
|
||||
|
||||
int iso8601_parse(const char *buf, size_t len, double *out) {
|
||||
iso8601_scanner s = {buf, buf + len};
|
||||
int64_t days, carry = 0;
|
||||
int extended, offset = 0, sep;
|
||||
double secs = 0;
|
||||
|
||||
if (!parse_date(&s, &days, &extended))
|
||||
return 0;
|
||||
|
||||
sep = scan_peek(&s);
|
||||
if (sep == 'T' || sep == 't' || sep == ' ') {
|
||||
s.p++;
|
||||
if (!parse_time(&s, extended, &secs, &carry) || !parse_offset(&s, &offset))
|
||||
return 0;
|
||||
}
|
||||
if (!scan_at_end(&s))
|
||||
return 0;
|
||||
|
||||
*out = (double)(days + carry) * 86400.0 + secs - offset;
|
||||
return 1;
|
||||
}
|
||||
@@ -0,0 +1,14 @@
|
||||
#ifndef JQ_ISO8601_H
|
||||
#define JQ_ISO8601_H
|
||||
|
||||
#include <stddef.h>
|
||||
|
||||
/*
|
||||
* Parses [buf, buf + len) as an ISO 8601 date and time of day.
|
||||
*
|
||||
* On success stores the number of seconds since the Unix epoch in *out and
|
||||
* returns 1. On failure returns 0 and leaves *out untouched.
|
||||
*/
|
||||
int iso8601_parse(const char *buf, size_t len, double *out);
|
||||
|
||||
#endif /* JQ_ISO8601_H */
|
||||
+105
@@ -1921,6 +1921,111 @@ last(range(365 * 67)|("1970-03-01T01:02:03Z"|strptime("%Y-%m-%dT%H:%M:%SZ")|mkti
|
||||
null
|
||||
[2037,1,11,1,2,3,3,41]
|
||||
|
||||
# fromdateiso8601 accepts the calendar, ordinal and week date forms,
|
||||
# in both the extended and the basic representation
|
||||
[("2015-03-05T23:51:47Z", "20150305T235147Z", "2015-064T23:51:47Z", "2015064T235147Z", "2015-W10-4T23:51:47Z", "2015W104T235147Z") | fromdateiso8601]
|
||||
null
|
||||
[1425599507,1425599507,1425599507,1425599507,1425599507,1425599507]
|
||||
|
||||
# fractional seconds, with either decimal sign
|
||||
[("2015-03-05T23:51:47.25Z", "2015-03-05T23:51:47,25Z", "20150305T235147.25Z") | fromdateiso8601]
|
||||
null
|
||||
[1425599507.25,1425599507.25,1425599507.25]
|
||||
|
||||
# UTC offsets; an absent offset means UTC
|
||||
[("2015-03-06T08:51:47+09:00", "2015-03-06T08:51:47+0900", "2015-03-06T08:51:47+09", "2015-03-05T18:21:47-05:30", "2015-03-05T23:51:47z", "2015-03-05T23:51:47") | fromdateiso8601]
|
||||
null
|
||||
[1425599507,1425599507,1425599507,1425599507,1425599507,1425599507]
|
||||
|
||||
# the date and the time may be separated by "T", "t" or a space
|
||||
[("2015-03-05t23:51:47Z", "2015-03-05 23:51:47Z") | fromdateiso8601]
|
||||
null
|
||||
[1425599507,1425599507]
|
||||
|
||||
# the W, T and Z designators may be written in lower case
|
||||
[("2015-w10-4t23:51:47z", "2015w104t235147z", "2015-03-05t23:51:47z") | fromdateiso8601]
|
||||
null
|
||||
[1425599507,1425599507,1425599507]
|
||||
|
||||
# expanded years carry a sign and exactly six digits
|
||||
[("+012015-03-05T23:51:47Z", "-000001-01-01T00:00:00Z", "+0020150305T235147Z", "+2025-03-05", "+02025-03-05", "+0002025-03-05") | try fromdateiso8601 catch "ERR"]
|
||||
null
|
||||
[316995119507,-62198755200,1425599507,"ERR","ERR","ERR"]
|
||||
|
||||
# "24:00:00" is midnight of the following day, at any precision
|
||||
[("2015-03-05T24:00:00Z", "2015-03-05T24:00Z", "2015-03-05T24Z") | fromdateiso8601 | todate]
|
||||
null
|
||||
["2015-03-06T00:00:00Z","2015-03-06T00:00:00Z","2015-03-06T00:00:00Z"]
|
||||
|
||||
# leap seconds are dropped rather than rejected
|
||||
"2015-06-30T23:59:60Z" | fromdateiso8601 | todate
|
||||
null
|
||||
"2015-06-30T23:59:59Z"
|
||||
|
||||
# 2015 starts on a Thursday, so it has 53 ISO weeks
|
||||
"2015-W53-1T00:00:00Z" | fromdateiso8601 | todate
|
||||
null
|
||||
"2015-12-28T00:00:00Z"
|
||||
|
||||
# 2016 is a leap year starting on a Friday: 366 days, but only 52 ISO weeks
|
||||
[("2016-366T00:00:00Z", "2016-W52-7T00:00:00Z") | fromdateiso8601 | todate]
|
||||
null
|
||||
["2016-12-31T00:00:00Z","2017-01-01T00:00:00Z"]
|
||||
|
||||
# fromdateiso8601 is not limited by the range of a 32-bit time_t (#3276)
|
||||
fromdate
|
||||
"2038-01-19T03:14:08Z"
|
||||
2147483648
|
||||
|
||||
# the time may drop its seconds, or its minutes and seconds
|
||||
[("2015-03-05T23:51Z", "2015-03-05T23Z", "20150305T2351Z", "20150305T23Z") | fromdateiso8601]
|
||||
null
|
||||
[1425599460,1425596400,1425599460,1425596400]
|
||||
|
||||
# a decimal fraction belongs to the lowest component present
|
||||
[("2015-03-05T23:51:47,5Z", "2015-03-05T23:51,5Z", "2015-03-05T23,5Z") | fromdateiso8601]
|
||||
null
|
||||
[1425599507.5,1425599490,1425598200]
|
||||
|
||||
# a date on its own is midnight UTC
|
||||
[("2015-03-05", "20150305", "2015-064", "2015064", "2015-W10-4", "2015W104") | fromdateiso8601]
|
||||
null
|
||||
[1425513600,1425513600,1425513600,1425513600,1425513600,1425513600]
|
||||
|
||||
# the zone designator belongs to the time, so a bare date cannot carry one
|
||||
[("2015-03-05Z", "2015-03-05+09:00", "2015-03-05T") | try fromdateiso8601 catch "ERR"]
|
||||
null
|
||||
["ERR","ERR","ERR"]
|
||||
|
||||
# the date must be complete, and a time must not be empty
|
||||
[("2015", "2015-03", "2015-W10", "2015-03T00:00:00Z", "T23:51:47Z", "T23:51:47", "23:51:47", "") | try fromdateiso8601 catch "ERR"]
|
||||
null
|
||||
["ERR","ERR","ERR","ERR","ERR","ERR","ERR","ERR"]
|
||||
|
||||
# the date and the time must agree on whether separators are used,
|
||||
# and "YYYYMM" is ambiguous with "YYMMDD"
|
||||
[("2015-03-05T235147Z", "20150305T23:51:47Z", "201503T00:00:00Z", "2015-03-05T2351Z", "20150305T23:51Z", "20150305T23205Z") | try fromdateiso8601 catch "ERR"]
|
||||
null
|
||||
["ERR","ERR","ERR","ERR","ERR","ERR"]
|
||||
|
||||
# out-of-range components are rejected
|
||||
[("2015-02-30T00:00:00Z", "2015-13-01T00:00:00Z", "2015-00-01T00:00:00Z", "2015-366T00:00:00Z", "2016-W53-1T00:00:00Z", "2015-W10-8T00:00:00Z", "2015-W00-1T00:00:00Z", "2015-03-05T25:00:00Z", "2015-03-05T24:00:01Z", "2015-03-05T24:01Z", "2015-03-05T23:60:00Z", "2015-03-05T23:51:47+24:00", "2015-03-05T23:51:47+09:") | try fromdateiso8601 catch "ERR"]
|
||||
null
|
||||
["ERR","ERR","ERR","ERR","ERR","ERR","ERR","ERR","ERR","ERR","ERR","ERR","ERR"]
|
||||
|
||||
# the input must be exactly a datetime: no surrounding whitespace
|
||||
[(" 2015-03-05T23:51:47Z", "2015-03-05T23:51:47Z ") | try fromdateiso8601 catch "ERR"]
|
||||
null
|
||||
["ERR","ERR"]
|
||||
|
||||
try fromdateiso8601 catch .
|
||||
123
|
||||
"fromdateiso8601 requires string inputs"
|
||||
|
||||
try fromdateiso8601 catch .
|
||||
"not a date"
|
||||
"date \"not a date\" is not a valid ISO 8601 datetime"
|
||||
|
||||
# module system
|
||||
import "a" as foo; import "b" as bar; def fooa: foo::a; [fooa, bar::a, bar::b, foo::a]
|
||||
null
|
||||
|
||||
Generated
+8
@@ -747,6 +747,14 @@ fromdate
|
||||
"2015-03-05T23:51:47Z"
|
||||
1425599507
|
||||
|
||||
fromdateiso8601
|
||||
"2015-03-06T08:51:47.25+09:00"
|
||||
1425599507.25
|
||||
|
||||
map(fromdateiso8601)
|
||||
["20150305T235147Z","2015-064T23:51:47Z","2015-W10-4T23:51:47Z"]
|
||||
[1425599507,1425599507,1425599507]
|
||||
|
||||
strptime("%Y-%m-%dT%H:%M:%SZ")
|
||||
"2015-03-05T23:51:47Z"
|
||||
[2015,2,5,23,51,47,4,63]
|
||||
|
||||
@@ -1,10 +1,5 @@
|
||||
# See tests/jq.test and the jq manual for more information.
|
||||
|
||||
# Regression test for #3276 (fails on mingw/WIN32)
|
||||
fromdate
|
||||
"2038-01-19T03:14:08Z"
|
||||
2147483648
|
||||
|
||||
# %e is not available on mingw/WIN32
|
||||
strftime("%A, %B %e, %Y")
|
||||
1435677542.822351
|
||||
|
||||
Reference in new issue
Block a user