alt Hacker NewsWe should just rewrite everything in Rust, especially bloated Python code, and the world would be a better place. ;) Disclosure: I'm a Rust advocate!
Replies
SOLAR_FIELDS • today at 7:59 PM
1 year ago everyone would have called you insane for suggesting this. Now we all shrug and say yeah maybe we can do this and it’s actually a good idea?
weberer • today at 8:09 PM
We should rewrite all Rust code in Python. Not for any technical reason. I'm just sick of the Rust cult at this point.
Both the example libraries compared (tokenizers and tiktoken) are Rust-based with Python bindings. There's just a few levers in Rust that can speed it up even more particularly with LLM assistance as the AI Use Discloure here notes:
> Final profiling stages and the last ~4x worth of performance from eliminating branching and improving the pretoken cache hierarchy