alt Hacker NewsIs there a gamechanger around the corner to reduce DRAM requirements?
Replies
stymaar • today at 8:45 PM
n-gram per-layer embeddings[1][2] might be it.
[1] https://sebastianraschka.com/llm-architecture-gallery/per-la...
[2]: See DS 4.1-Flash and Qwen-3.8-Next.
➕ show 1 reply
You could always stream from SSD storage. Especially effective if you get a cheap old-gen HEDT with lots of PCIe slots to add NVMe storage to and reasonable overall PCIe bandwidth.