logoalt Hacker News

petutoday at 7:56 AM1 replyview on HN

It's larger than previous V4 Flash.

  552B in ~FP4, 306GB.   
  196B of FP8 Engrams, another 204GB, not necessary to keep in RAM.  
  KV cache sees another 4x size reduction, just 900MB for 1M.  
So 384GB needed for a chance of achieving useful speeds. Three Sparks or quad RTX PRO 6000.

Replies

npodbielskitoday at 8:52 AM

Or two gorgon halos?