Interfacing with the generated lexer

The easiest way of interfacing with a lexer is to embed an object of class LexerBuffer, IOLexerBuffer, or IOLexerStack within a syntax class². We can then define the tokenization method get_token using the matchscan statement described in section 3.1.5.