deepseek v3 and r1 are both 700B models, who has that much memory to run the model locally these days?
Exolabs claims they can distribute the compute over many machines to use memory in aggregate: https://github.com/exo-explore/exo
Maybe there is enough memory in many machines.
alt Hacker News
Exolabs claims they can distribute the compute over many machines to use memory in aggregate: https://github.com/exo-explore/exo
Maybe there is enough memory in many machines.