Due to personal failings, my main takeaway from this piece is that Regex is under attack -- I will not stand by as CS theory blorbo is maligned so! Of course there's an undeniable elegance to Lisp in all cases, and seeing Lisp in use today feels like seeing someone unsheathe Valerian steel. But still, the fact remains: regex is vastly underappreciated by the standard engineer, largely because a few of the most common implementations drop some of the most important features. Namely, composition.
I have no easy way to express this other than to plug my recently-released OSS, namely a library I wrote to implement "RegexStores" in Python[1] to compile complex, composed regexes upfront from a custom DSL. I'll link a representative usecase below[2], which I think drives home two things:
1. Regex deserves to be composed using named groups, at the very least! Most people aren't even aware that you can define named groups upfront and then reference them by name throughout, even 'capturing' them multiple times in one match.[3] Even when you do write patterns that use subroutines, the ergonomics of actually accessing, say, the three matched substrings for the `username` group in your single match is finnicky at best (without some magic[4]...).
2. Regex needs high quality syntax highlighting. The linked page is hopefully skimmable in a python sense, but trying to decipher the individual patterns in GitLab would be tough even for me, and I wrote the darn things. It's kinda awkwardly sized, but this screenshot drives home the basics of the point -- note the bright yellow named groups, which are basically subroutine invocations as discussed above: https://i.imgur.com/NllqM6S.png
Sorry to quasihijack the thread. Hopefully the fact that all this stuff has never been announced or published anywhere is proof enough of my good intentions -- that is, to defend my one n' only :)
Most of this functionality was conceived and implemented in a rage after I found out that Rust's main (only?) regex library avoids the possibility of infinite loops by just dropping half the features of the language, so excuse the jank. Even the halting problem seems solvable when you're willing to make moves like that!
[1]: https://gitlab.com/doering-ai/libs/basis/#regular-expression...
[2]: https://gitlab.com/doering-ai/apps/wiki-parse/-/blob/main/wi...
[3]: I wrote up an overview of advanced regexes in Python a while back, but was hit by the agential engineering bus before I had the time to polish and release it. Some may find it interesting, if overly long -- the aforementioned subroutines are covered under `3.1`, and repeated captures under `6.2`: https://gitlab.com/doering-ai/libs/basis/-/blob/main/docs/re...
[4]: https://gitlab.com/doering-ai/libs/basis/-/blob/main/my/rege...
I think you make a great point in highlighting the value of Regex. I don't want to undermine it at all, although some use-cases are just not a good fit for it (even if you can horseshoe things). I think a higher-level abstraction for regex is super welcome, and I quite admire what you are doing there, thanks a lot for sharing. I do think regex's indeed quite a lot more difficult to grasp when complexity rises compared to grammars or other approaches.