alt Hacker NewsReally? I'm surprised. What do you use it for?
My read on Diffusion LLMs has been that it's cool tech, but only useful for things like like code autocomplete or editing a block of text.
They don't beat autoregressive models on quality per compute, assuming you are serving lots of users and can batch requests.
MoA schemes that use proposers/task-separation/supervisory roles can still benefit from very fast diffusion models like mercury, the overall speed doesn't have to be bottle-necked by the slowest model -- that depends on the arrangement it's deployed in.
but to your point , yeah diffusion models can be kind of brain dead without proper coaxing.
my ideal arrangement is a smarter/bigger model supervising lots of proposals from diffusion models. The diffusion models spit out a big proposal each, the bigger model can vet something as minor as the headline or filename for each proposal, time still ends up being saved.