Arborium: Tree-sitter code highlighting with Native and WASM targets

https://fasterthanli.me/articles/my-gift-to-the-rust-docs-te... is a better link (the author's article about this rather than the artifact produced)

Daaang! On the one hand, as someone developing a code-example custom element[0] that includes highlighting, this is kind of a "so close, but not quite" situation where I really wish this was something I could use for that, but it's probably too heavy to ship around for something so "simple" (as far as users' default expectation; not implying that highlighting text is simple).

But then, on the other hand, I had given up on a scratch code editor for a game development project I'm working on, and just loosely wrapped up the monaco editor which I'm guessing is going to be pretty bare when I actually get around to trying to code with it, in browser (I'm aware that it is robust, but from what I gather, a lot of its features come from third-party dev as a way to keep the core functionality pure). Given that I want to be able to script in C# (aside from just js/ts), I was sure I was going to have to figure something complicated out.

But, honestly, I think this solves all of my most concerning issues! What a sweet little library!

[0] https://magnitce-code-example-e81613.gitlab.io/ (please excuse the unfinished-ness; I'm working on a JSDoc-to-documentation library that automates the documentation for me so there are minor issues, like the install text not changing on selection)

Completely appalled to learn that docs.rs lets you inject any html/css/js you want into the live site (on pages documenting your crate). I love the flexibility but shudder at the security hole the size of, oh, I don’t know, the Grand Canyon.

It’s not a new discovery, I just didn’t know docs.rs (intentionally) wasn’t blocking this. Cf https://docs.rs/pwnies/0.0.13/pwnies/

(This all makes more sense if you read the blog post instead of the direct arborium link: https://fasterthanli.me/articles/my-gift-to-the-rust-docs-te...)

I'm currently building an online 3D-Editor that supports OpenSCAD and Python as the input language.

The ease of use to highlight static text via Arborium seems nice:

    <script src="arborium.iife.js"></script>
    <pre><code class="language-python">
        def hello(name):
            print("Hello " + name);
    </code></pre>

And how would one add SCAD support?

There's also a pure-Rust implementation of a syntax highlighter, which uses TextMate/SublimeText grammars: https://lib.rs/syntect

I've been toying around with tree-sitter and have seen the potential for a proper, non-regex based highlighter. It's really powerful because it actually parses the text into an AST. With the AST it's possible for example to make variables the same colour everywhere. A function passed as a parameter could be highlighted as a function even in the parameter list.

I'm happy to see that tree sitter highlighting on the web is finally a thing. This seems really solid although the bundle size is a lot.

If you haven't tried building a grammar with tree-sitter, I highly recommend you do so. It's incredibly fun once you get into a flow state. The docs call it a zen-like experience, and that's a perfect way to describe it. It's so, so good.

The sponsorships achieved by the author is admirable - they really make a lot of valuable oss.

https://github.com/sponsors/fasterthanlime

Treesitter is fantastic. It has builtin support in nvim, and there are a lot of plugins that make use of it.

My favorite is nvim-treesitter-textobjects which gives you dozens of new targets for vim motions, such as a function call or the condition of a loop.

I had to wait for about 10 seconds for it to load on my crappy mobile connection.

They also load 1 mb of fonts. In total, this page is close to 3 mb.

Also, when you select a language, the grammar file gets downloaded twice.

Tree-sitter:

Tree-sitter is a parser generator tool and an incremental parsing library. It can build a concrete syntax tree for a source file and efficiently update the syntax tree as the source file is edited.

Great project, I really love tree-sitter, recently I added a ini variant config profile support to my app, and just use gemini to write a grammar and combine it with another great project called runestone to support highlight the config profile, the total progress is quite smooth.

This is cool! How does it compares with Shiki or Highlighter.js in terms of performance?

Sorry, but I can't understand what this actually is. A library, a stand-alone tool, a Rust crate? What users does it target? Text editors, website creators?

The get started section seems to be broken or missing content.

This is cool, was looking for something similar