alt Hacker NewsHe's also built a closed-source LLM inference engine, which he's been maintaining since the GPT-2 days: https://bellard.org/ts_server/ and https://textsynth.com/
He's also built a closed-source LLM inference engine, which he's been maintaining since the GPT-2 days: https://bellard.org/ts_server/ and https://textsynth.com/
I used to play around with Textsynth, but not being OSS killed the appeal for me once llama.cpp came around.