Thank you. Which is currently the most capable version running reasonably fast on a 3090 (24GB of VRAM)?
The Llama distilled version Q4_K_M should be reasonably fast and good!!
alt Hacker News
The Llama distilled version Q4_K_M should be reasonably fast and good!!