alt Hacker NewsI am curious if it's possible to adjust this to use more RAM, as i've got a machine with 64GB RAM and 24GB VRAM. Or perhaps I could run Gemma/Qwen on the GPU and have GLM-5.2 delegate smaller tasks to it. It might take some retraining of GLM-5.2
I'm also curious if you can speed this up by using many disks in parallel to increase bandwidth.
>SSD Wear Warning
> Cold starts are heavy on random reads (~11 GB/token). Reads themselves are safe, but the OS page cache can generate writes. Heavy use may accelerate wear on cheaper SSDs. Use with caution and monitor your drive health.
Hmm, maybe a safe way to do this would be to make a separate partition for the model weights, and set them to read-only? Not sure how the page cache works, if it's like per partition or per disk. If it's per disk, maybe you could have a read-only data.iso formatted as a partition and mount it as a disk?
Replies
> OS page cache can generate writes
Is this a hallucination? What am I missing? Why would heavy reads generate writes?
I have a small laptop. If you have more disks available, you could really do some testing. When you have some benchmarks, submit a pull request or issue so we can maybe work on them. We are really happy for contribute!