logoalt Hacker News

tormeh • 10/12/2024 • 0 replies • view on HN

Depends. Good models are big, and require a lot of memory. Even the 4090 doesn't have that much memory in an LLM context. So your GPU will be faster, but likely can't fit the big models.