alt Hacker News(author of the blog post here)
For me, the hardest part was virtualizing GPUs with NVLink in the mix. It complicates isolation while trying to preserve performance.
AMA if you want to dig into any of the details.
Replies
spwa4 • last Thursday at 8:17 PM
Would it be possible to implement "virtual memory" for a GPU this way? Let's say you have GPUs at 30% utilization, but memory limited. Could you run 2 workloads by offloading the GPU memory when not in use?
➕ show 1 reply
Isn't SR-IOV a thing with these big GPUs? Or, is it that you're not concerned with fractional granularity?