Very nice indeed. Model weights in RAM blocks distributed all over a big FPGA: should be super helpful at minimizing RAM bandwidth bottlenecks. To say nothing of latency.

But model(s) implemented are clearly too small to be useful as a 'chat partner'. Tried a couple of sentences - replies is just some gibberish coming out.

This really needs a bigger FPGA, or some other application(s) where a tiny LLM does actually useful work. Barring that, generated tokens/sec is kind of a meaningless measure imho.

I am also very cautious with people who tell me something impossible when I can trust my engineering skills and get a good sense that there is potentially a good outcome. In my experience, it simply means they don’t know how to do it, or are frustrated they couldn’t do it themselves and get into the spotlight.