We're serving it around 150-200tok/s (uses our new speculative decoding implementation on ...

zackangelo • today at 7:05 PM • 4 replies • view on HN

We're serving it around 150-200tok/s (uses our new speculative decoding implementation on a DFlash2 draft model).

https://mixlayer.com, LAUNCH-Q38-27B gets you $5 in credits if you want to kick the tires.

I don't see any kind of input cache discount listed on your pricing page. Do you offer that, or is all input priced the same?

danielklnstein • today at 7:29 PM

I tried in your playground and got 14.2 tok/s?

➕ show 2 replies

bookernath • today at 7:43 PM

This feels great

alt Hacker News