Jeffrey Kegler, the author of Marpa, was kind enough to suggest a few changes to the parser from the previous blog post. Two of the three regular expressions were replaced with calls to the Marpa recognizer. The first one was replaced with a call to the pause_span method, which returns the start and length of the lexeme that caused the pause. Passing these two values to the literal method which returns the substring from the input string....
I have been thinking on and off about how to make Marpa parse a template language. I tried many different ways to create a tokenizer for this parser, but until this week I couldn’t get something reliable. The week I started with the tokenizer from the example code from Jeffrey Kegler’s blog post about how to develop a parser iteratively. This got me on the right track to create my parser....
I just wrote a small program to parse and generate a parser that parses Marpa. Marpa itself doesn’t include such a program for good reasons I assume. Or if it exists I couldn’t find it. But as this is a small program it only generates a part of the parser. It is also not bootstrapping. The repository contains a small tokenizer written in Perl. It tokenizes a line of text and passes this on to the token recognizer, which in turn calls the grammer....