logoalt Hacker News

xfalcoxyesterday at 6:20 PM1 replyview on HN

We have vLLM for running text LLMs in production. What is the equivalent for this model?


Replies

mh-yesterday at 7:42 PM

I would say there's isn't an equivalent. Some people will probably tell you ComfyUI - you can expose workflows via API endpoints and parameterize them. This is how e.g. Krita AI Diffusion uses a ComfyUI backend.

For various reasons, I doubt there are any large scale SaaS-style providers operating this in production today.