alt Hacker NewsI was under the impression that when you're streaming the weights from disk because the full model won't fit in memory, that it is solely reading from the SSD, not writing, so it wouldn't be causing wear on your SSD.
Replies
NAND[0] has a fun thing called "read disturbance" where repeated reads from disk will eventually flip 0s to 1s. You have to erase and rewrite the block before the bits flip[1], or you lose the data, but doing so is the same amount of wear as a write.
[0] I heard this being an issue with TLC, I don't know if it also applied to MLC or SLC.
[1] I suspect in practice they use an error correction code and rewrite blocks that read with corrected errors.
It is and it doesn't. You only get into disk writes if the system starts paging out to disk.
You'd need your OS to support, and be configured to use, a disk mounting option that disables file access timestamps, otherwise reads ARE writes.