logoalt Hacker News

estebanktoday at 5:23 PM0 repliesview on HN

I think you meant TokenStream. They are trees, behind the scenes, because matching delimiters happens early on between lexing and parsing. By the time the rustc_proc_macro::TokenStream is exposed, the rustc_ast::tokenstream::TokenTree is hidden to the proc macro API.

https://doc.rust-lang.org/stable/nightly-rustc/rustc_ast/tok...

https://doc.rust-lang.org/stable/nightly-rustc/rustc_proc_ma...