alt Hacker NewsThanks for your insight, much appreciated!
However, regarding this:
> Much like operator overloading in other languages, catcode changes in TeX can indeed be misused and are sometimes confusing, but they're also a pretty useful solution to problems that would otherwise be tricky to solve.
I'm sorry, but I've never seen overloading of such fundamental characters like the comment character or escape character anywhere. Or at least if you use these characters inside a string, it's pretty clear that the string context is special. In LaTeX I have no way of knowing which catcodes a macro has modified without essentially parsing the entire thing, which breaks syntax highlighters and language servers (something that increases quality of life in other languages substantially), because the compile times are prohibitive. The decision to let users redefine %, \ and literally every character seems like a really, really bad idea to me.
Other languages and syntaxes seem to do just fine, so I'm not sure what you mean by tricky to solve.
> most of these are due to the fact that LaTeX is 40-year-old software built on a 50-year-old engine, and has remained backwards-compatible with documents throughout that entire time
I realize that, and I appreciate what LaTeX (and by extension TeX) has done. It's a giant in sciences and the software world, of absolutely critical importance, but still. We learned a lot of lessons about writing software in the last 50 years, and Typst is applying these from the ground up. Unfortunately I don't have a lot of confidence that LaTeX can be modernized.
> I'm sorry, but I've never seen overloading of such fundamental characters like the comment character or escape character anywhere.
I believe that Racket [0], Mathematica [1], Raku [2], and Rust [3] let you assign arbitrary meanings to most symbols, but these are indeed much more restricted than TeX is (and for good reason). But the issue is really more that TeX barely supports lexical scoping, and that it lets you change the global catcodes at any point in the document, since changing the meanings of characters before any code runs or only in the middle of a scope is pretty useful.
> Or at least if you use these characters inside a string, it's pretty clear that the string context is special.
Verbatim is essentially equivalent to strings in other languages, and it mostly works pretty well, aside from the huge problem that it's impossible to nest it or pass it as an argument to most macros.
> In LaTeX I have no way of knowing which catcodes a macro has modified without essentially parsing the entire thing
Agreed, this is pretty annoying, but the only consolation is that most documents don't change their catcodes very often (since it's usually a pretty terrible idea).
> Other languages and syntaxes seem to do just fine, so I'm not sure what you mean by tricky to solve.
Texinfo [4] is what I was mainly thinking of, since that is able to completely redefine TeX's syntax without needing to manually implement parsing itself (which would be the best strategy today, but was less feasible back when computers were much slower). Similarly, active characters are pretty useful (this is how ~ is defined to insert a non-breaking space, and is also useful for faux-Markdown [5]).
> We learned a lot of lessons about writing software in the last 50 years, and Typst is applying these from the ground up.
Yup, I first learned about Typst on the first day that it was released, and even then I thought that it had a good chance of suceeding, since it solved the problems that most users had (bad error messages and slow compile times), whereas the other TeX competitors focused on things like better typesetting quality, better extensibility, or easier programability, which most users don't care about at all.
I would be personally a little disappointed if Typst replaced LaTeX, but until that happens, I definitely hope that it continues to do well.
> Unfortunately I don't have a lot of confidence that LaTeX can be modernized.
There are lots of other non-LaTeX TeX formats that are quite modern (ConTeXt [6] is my personal favourite, but OpTeX [7] is nice too), and even LaTeX itself has improved quite a bit over the last 5 years or so [8]. But yeah, it's modernization process is still much slower than nearly any other piece of software, so I am also worried that this may end up being too little too late.
[0]: https://docs.racket-lang.org/guide/language-get-info.html
[1]: https://reference.wolfram.com/language/Notation/tutorial/Not...
[2]: https://docs.raku.org/language/slangs
[3]: https://doc.rust-lang.org/book/ch20-05-macros.html#function-...
[4]: https://en.wikipedia.org/wiki/Texinfo#Texinfo_source_file
[5]: https://tex.stackexchange.com/a/236457/270600
[6]: https://www.ctan.org/pkg/context
[7]: https://petr.olsak.net/optex/
[8]: https://www.latex-project.org/news/latex2e-news/ltnews.pdf#s...