Talk:Raku Programming/Grammars

I've got a few problems about the content of this chapter:

1. The section "Grammars" describes them essentially as syntactical sugar for regexps, while they are intended to understand a wider class of strings than regexps

2. Subsection "Tokens" says "Tokens are are regex that don't backtrack" (btw., "are" is duplicated). If this were true, the code

token protocol { 'http'|'https'|'ftp'|'file' }

in the next subsection should be unable to parse "https://foo.bar.com", since the non-backtracking token would get stuck on "http" and the grammar would then miss the expected "://", running into an unexpected 's'. This is a problem not only of regexps, but even of LL-Parsers.

Unfortunately I don't know (my parrot is still compiling) whether the code works (and therefore the subsection "Tokens" is wrong) or vice versa. That's why I wrote a comment instead of changing the text. Ralf Muschall (talk) 10:39, 27 August 2009 (UTC)--

In the meantime, I did some tests and readings. There are the keywords "token", "rule" and "regex". "regex" does what one would naively expect, the other two suppress backtracking. "rule" in addition means that whitespace is significant, i.e.

rule URL { '://' }

would not work. The grammar is a PEG (see http://en.wikipedia.org/wiki/Parsing_expression_grammar), i.e. left-recursions must be eliminated by left-factoring, but this is somewhat more powerful than a CFG. The simplest way to fix this section would probably be to grab stuff from the Wikipedia articles PEG and http://en.wikipedia.org/wiki/Perl_6_rules, and to use a few tested examples. Ralf Muschall (talk) 08:30, 28 August 2009 (UTC)--

I have fixed the examples so they at least mostly work, although I am not sure how good the examples for rules are. BTW. You are correct about https not working but the version I have given in the protos section should as the longest token will be chosen. I have some more complete code for a basic mathematical expression parser which might make better examples (after being simplified) and would also demonstrate actions, if people think it would be more appropriate: Twright (discuss • contribs) 14:14, 15 May 2011 (UTC)