But you could grab a 5090, and paired with some DRAM for MoE offloading of bigger models, and be a happy camper with 1.8TB/s of memory bandwidth.

Or just use Luna honestly. Worth considering if you’re ok with hosted APIs.