Dense Arena Interning: The Engine of Compiler Performance

A collection of thoughts I had while reading the article:

0. Overall, a nice article.

1. There are several places where the (LLM assisted?) writing is a bit odd. For example, the code for hash table insertion is irrelevant. The "“Un-freeable” Nature of Arenas" paragraph sets up a problem that I expect the next section to resolve, but the next section goes in a different direction.

2. It feels like the author is on the verge of rediscovering some classic PL concepts. E.g. tries or regular expressions for faster lexing (why would you ever do strcmp in a loop in a lexer?); de Bruijn indices for disambiguating lexically scoped variables; strong normalization in type systems to canonicalize types.

3. I was expecting some benchmark results.

So, it's like converting to symbols in lisps. EQ on symbols is also a single word comparison.

An interesting part is where you want to have maps from these.

Common python strings are interned to speed things up as well. Maybe this way of interning could benefit cpython.