The only thing that compares to this is probably Mac mini with MLX models.
Radeon 9700 pro or intel arc b70 (both $1000-1400, 32GB, 650GB/s bandwidth), or ryzen AI max 390 (more vram, less bandwidth)
The local inference space is pretty good nowadays.
alt Hacker News
Radeon 9700 pro or intel arc b70 (both $1000-1400, 32GB, 650GB/s bandwidth), or ryzen AI max 390 (more vram, less bandwidth)
The local inference space is pretty good nowadays.