alt Hacker NewsThere is a down to earth guy on YT that performs a series of tests against LLMs running on non-god-tier commodity hardware. He will likely be testing this soon enough.
https://www.youtube.com/@lukesdevlab
I don't know if that is what you are looking for or not and as always your experiences may be different.
Replies
yassa9 • today at 4:14 PM
thaaanks man, this channel seems really informative, although < 10K subs only !
➕ show 1 reply
So much potential for that channel. He's got a nice range of tests and a no nonsense presentation style.
However, watching tests of heavily quantized models that weren't designed for it (non-QAT) is frustrating. There's no way to tell if the actual model fails because it's dumb or if the lobotomy made it that way.