logoalt Hacker News

Show HN: Yantra – an LALR(1) parser generator for C++

23 points • by renjipanicker • today at 2:30 AM • 14 comments • view on HN

Yantra is a C++ parser generator: lexer, parser, and AST walker all generated from one tool. It builds the whole AST first, then walks it.

Most LALR parser generators (Yacc, Bison, Lemon) run your semantic actions during parsing, as each rule reduces, bottom-up.

That means at the time a rule's action runs, you don't yet know what its parent looks like. This pushes a lot of grammars toward hand-built AST classes and a separate walking pass whenever you need to look ahead into siblings or defer a decision until more context is available.

On the other hand, Yantra always builds the whole AST first, then walks it top-down in a separate pass, calling your semantic actions as it goes. A parent rule's action can run before its children are visited.

A single grammar can define more than one walker. For example, one that emits C++, another that emits Java, from the same parse. The AST and the walker classes are both generated for you.

A small example (full version, with compile commands, in the README):

  start := expr;

  expr := expr(a) PLUS expr(b)
  %{
      std::cout << "Adding" << std::endl;
  %}

  expr := NUMBER(N)
  %{
      std::cout << "Number: " << N.text << std::endl;
  %}

  NUMBER := "\d+";
  PLUS := "\+";
  WS := "\s+"!;
Running this on "1 + 2 + 3" prints:

  Adding
  Number: 1
  Adding
  Number: 2
  Number: 3
The outer "Adding", the root of the tree, prints first, before either of its children. That's only possible because the whole tree exists before any action runs.

Some other things about it: integrated lexer with mode support (for things like nested comments), an optional amalgamated single-file output mode with a generated main(), C++23, MIT licensed.

It's young (0.5.1, pre-1.0) and single-maintainer, so treat it as early. I'd rather know what breaks than have it look more finished than it is.

Known gaps are listed at https://github.com/TantrixAuto/yantra/blob/main/docs/known_l...

Repo: https://github.com/TantrixAuto/yantra

Feedback and questions are all welcome. I'll be around.


Comments

mingodad • today at 6:01 AM

For people interested on this topic I strongly recommend to also look at Ben Hanson https://github.com/BenHanson/parsertl17 and based on it I've created an online LALR(1) playground here https://mingodad.github.io/parsertl-playground/playground/ where you have around 350 non trivial. grammars to experiment (select one from the `Examples` dropdown and then click `Parse` to see a parse tree for the input in `Input`, it also generates EBNF to generate nice navigable railroad diagrams on https://www.bottlecaps.de/rr/ui .

userbinator • today at 4:14 AM

It's a little surprising to see new parser generators being written, long after the vast majority of compilers have already settled on recursive descent / precedence climbing (including https://news.ycombinator.com/item?id=49913192 , which is currently nearby on the front page.)

➕ show 4 replies
froh • today at 7:09 AM

do you intend to add sth like python bindings, especially for the AST and walker classes?

then Yantra would become great to parse well-known structured data from python at low-level speed.

kazinator • today at 3:55 AM

Yacc has mid-rule actions which can be used to propagate information from left siblings to right siblings, as well as to children (embedded nonterminal symbols).

Of course, it's not the same as having the parse tree all done from a previous pass and just walking it to do semantics.

➕ show 1 reply
signa11 • today at 6:08 AM

isn't this close to what ragel does ?

fithisux • today at 4:36 AM

Congratulations. We need more of these tools. I'll give it a try.

➕ show 1 reply