alt Hacker Newsi thought my post was already too long to include this, but to your point, you can run AI inference in this setup and the performance can be pretty good.
Replies
shmeeed • yesterday at 7:36 AM
I appreciate you making the post not about AI.
colechristensen • last Friday at 10:52 PM
Yeah it is obviously actually useful for AI inference over which the pcie speed isn't particularly important and a single board computer gets you a small system.
There are definitely some use cases where it works out, others where it doesn't; I spent a bit of time testing that side of things late last year: https://www.jeffgeerling.com/blog/2025/big-gpus-dont-need-bi...