AT_LEAST_ONE_SEP validating non seperated identifiers

[smydo]

Hi All

Hopefully someone can help me understand where my Rule definition is going wrong. I have the following sql based single condition rule definition (simplified for this discussion):

I expect this to allow for n number of chained values seperated by an arithmetic operator (ValueSeperator) which must be followed by a relational operator combined with another chain of n numbered values i.e. for this to be valid:

a + b +c = d + e +f

This does prove to be valid but I would also expect that the following will NOT be valid:

a + b a a a = 5

but it is! I expect that the AT_LEAST_ONE_SEP would find fault with the multiple 'a's that have no arithmetic operator seperating them. Checking the parser instance object in debug confirms that these 'a's are held as seperate 'ValueWithoutIn' identifier tokens within the singleCondition (each with the 'lhs label): I didn't think the parser would be 'greedy' and take multiple ValueWithoutIn tokens despite them being in the AT_LEAST_ONE_SEP construct. What am I missing here?

Thanks

[msujew]

Hey @smydo,

This does prove to be valid but I would also expect that the following will NOT be valid

What does validity mean for you in this context? Chevrotain parsers won't necessarily throw on invalid input (one that doesn't match the expected grammar), but save the error on the errors property of the parser instance. Have you tried inspecting those?

I didn't think the parser would be 'greedy' and take multiple ValueWithoutIn tokens despite them being in the AT_LEAST_ONE_SEP construct. What am I missing here?

The parser isn't necessarily greedy (the lookahead will fail on the AT_LEAST_ONE_SEP function), but the error recovery will likely try to continue parsing the input, even if it contains a few invalid parts. Is the error recovery even enabled for your parser?

[smydo]

Hi Mark

Thanks for the quick reply. My implementation of this is such that I'm checking for the existence of errors in the parserinstance but there are none for the 'a = b a a a = 5' string I parsed. If I parse something like '1=+1' I do get an error. Is there some other configuration of erros that I'm missing?

[smydo]

Sorry @msujew, to answer your question, error recovery was enabled but I have tried the 'errorRecovery: false' option in the parser instance constructor and there is no difference in parsing that nonsense string, there are no errors recorded in the parser instance object. I think this phenomenon of the parser 'skipping' the rogue identifiers and hence 'validating' the string is explained in this doc about fault tolerance , specifically Repetition Re-Sync:
https://chevrotain.io/docs/tutorial/step4_fault_tolerance.html#repetition-re-sync

, however it's not clear how to circumvent this behaviour if it doesn't suit your needs

[msujew]

That's very weird. If a = b a a a = 5 yields no parser errors, then it has to be a valid string as per your grammar. Can you perhaps share a reproducible example so I can probe deeper into it?

[smydo]

I must be doing somthing fundamentally wrong, I've included the parser code an lexer tokens files. I call the parser function like this:

parserFunction(<expressiontext.);
parserCode.txt
lexerTokens.txt

[msujew]

I can't really run it right now, but looking at the valueWithoutIn parser rule, it makes sense why a = b a a a = 5 can be valid in your grammar. When I simplify it a bit, the important bit can be seen:

$.RULE('valueWithoutIn', () => {
  $.AT_LEAST_ONE(() => {
    $.OR([
      ...
      { ALT: () => $.CONSUME(tokenVocabulary.Identifier) }
    ]);
  });
});

That effectively enables the rule to parse any list of identifiers which appear right after another.

[smydo]

Sorry again @msujew that was madness to think you'd engage with 2 files , try this apex app, you'll be able to see everything I mean:

[smydo]

Ah...! Thanks a million Mark, I think that's it!

[smydo]

https://apex.oracle.com/pls/apex/f?p=124813:2