alt Hacker NewsIs anyone here experimenting seriously with Diffusion for text generation? I’d love to learn about your experiences!
Replies
I've found the latency and pricing make Mercury 2 extremely compelling for some UX experiments focused around automated note tagging/interlinking. Far more than the Gemini Flash Lite I used before, it made some interactions nearly frictionless, very close to how old school autocomplete/T9/autocorrect works in a manner that users don't even think about the processes behind it.
Sadly, it does not perform at the level of e.g. Haiku 3.5 for tool calling, despite their own benchmarks claiming parity with Haiku 4.5, but it does compete with Flash Lite there too.
Anything with very targeted output, sufficient existing input and that benefits from a seamless feeling lends itself to dLLMs. Could see a place in tab-complete too, though Cursors model seems to be sufficiently low latency already.
It's being explored right now for speculative decoding in the local-LLM space, which I think is quite interesting as a use-case
https://www.emergentmind.com/topics/dflash-block-diffusion-f...
I have. It requires a distinct intuition compared to a normal language model. Very well suited to certain problems.
I've been playing with a Swift implementation of a diffusion language model (WeDLM), but performance is not yet acceptable and it still generates roughly from left-to-right like a language model (just within a sliding window rather than strictly token-by-token... but that doesn't matter when the sliding window is only like 16 tokens.)
https://www.inceptionlabs.ai/
This startup seems to have been at it a while.
From our look into it - amazing speed, but challenges remain around time-to-first-token user experience and overall answer quality.
Can absolutely see this working if we can get the speed and accuracy up to that “good enough” position for cheaper models - or non-user facing async work.
One other question I’ve had is wondering if it’s possible to actually set a huge amount of text to diffuse as the output - using a larger body to mechanically force greater levels of reasoning. I’m sure there’s some incredibly interesting research taking place in the big labs on this.