alt Hacker News

Predictive Speculative KV Replication for Bursty LLM Inference

17 points • by shreybirmiwal • yesterday at 7:55 PM • 1 comment • view on HN

https://github.com/jwlaboratory/bite-the-bullet

Comments

msp26 • yesterday at 10:58 PM

the hand drawn diagrams and highlights are charming

➕ show 1 reply