Microgpt in pure C hits 10M tps on Apple m5

> The most atomic way to train and inference a GPT in pure, dependency-free C.

What sense of the word "atomic" is meant here?

Check out this port of microgpt to C, posted 5 months ago. It got a 2500x speedup over the python version. https://github.com/moebiusV/cugpt

This is not an LLM obviously , it's just for generating random names. But interesting to think of the possibilities of truly tiny language models if there were connected together.

And the 5 years old AMD Ryzen 5 5600H is doing 7M?

Am I reading this right? Then I need to try this on Strix Halo

Honestly not sure this is impressive. I ported microgpt to zig as a learning exercise, then moved scalar engines to NEON/metal just to see what happened. Besides metal being slower (I probably did something wrong, but it could be due to the fixed costs of memory transfer into the GPU not being worth it due to the small model).

Anyways, it was also stupid fast, particularly compared to the python version. But I was pretty sure that's irrelevant to real production architectures!

Model is 4K parameters - I don't know enough about that size of model to know if this impressive or not.