On 2024-03-25 Mo 19:02, Andrew Dunstan wrote:
On Wed, Mar 20, 2024 at 11:56 PM Andrew Dunstan <andrew@dunslane.net> wrote:
> Thanks, included that and attended to the other issues we discussed. I think this is pretty close now.
Okay, looking over the thread, there are the following open items:
- extend the incremental test in order to exercise the semantic callbacks [1]
Yeah, I'm on a super long plane trip later this week, so I might get it done then :-)
- add Assert calls in impossible error cases [2]
ok, will do
- error out if the non-incremental lex doesn't consume the entire token [2]
ok, will do
- double-check that out of memory is an appropriate failure mode for
the frontend [3]
Well, what's the alternative? The current parser doesn't check stack depth in frontend code. Presumably it too will eventually just run out of memory, possibly rather sooner as the stack frames could be more expensive than the incremental parser stack extensions.
Just as a general style nit:
> + if (lex->incremental)
> + {
> + lex->input = lex->token_terminator = lex->line_start = json;
> + lex->input_length = len;
> + lex->inc_state->is_last_chunk = is_last;
> + }
> + else
> + return JSON_INVALID_LEXER_TYPE;
I think flipping this around would probably make it more readable;
something like:
if (!lex->incremental)
return JSON_INVALID_LEXER_TYPE;
lex->input = ...
Noted. will do, Thanks.
Here's a new set of patches, with I think everything except the error case Asserts attended to. There's a change to add semantic processing to the test suite in patch 4, but I'd fold that into patch 1 when committing.
cheers
andrew
--
Andrew Dunstan
EDB: https://www.enterprisedb.com