alt Hacker NewsBut you arent really running LLMs. You just say you are.
There is novelty, but not practical use case.
My $700, 2023, 3060 laptop runs 8B models. At the enterprise level we got 2, A6000s.
Both are useful and were used for economic gain. I don't think you have gotten any gain.
Yes a good phone can run a quantised 8B too.
Two A6000 is fast but quite limited in memory. It depends on the use case.