logoalt Hacker News

vidarh07/18/20251 replyview on HN

In 45 years of writing software, I've never before seen anyone call tokenizing a BASIC program compilation. It's decidedly not common usage.


Replies

WalterBright07/18/2025

I've been writing compilers for 45 years now. Tokenizing is a big part of every textbook on compilers. To resolve expressions (which are recursive in nature) it would have had to do more than just tokenizing. While this isn't hard at all, it's "parsing" which is also qualifying it as a compiler.

I.e. the basic program was lexing and parsing. It's a compiler. A very simple one, sure, but a compiler.

show 2 replies