> What's you opnion vs. SGLang in this regard?

No opinion yet, but I'm planning to temporarily switch and see how it goes. I really hope the grass is greener there, but I'm keeping my expectations in check for now.

I recently switched to using for some large volume inference with small, fine-tuned models and was surprised how nice and painless things were. vLLM seems to have a lot more gotchas and kludged together stuff once you get outside of anything straightforward.