alt Hacker NewsMy 1080ti is my local inference machine now. With the release of the Bonsai 27B models I can run a genuine dense model with usable context. GOAT card.
Replies
mrandish • yesterday at 7:21 PM
I have a 1080ti currently not plugged in. More info please.
➕ show 1 reply
What exact runtime are you running Bonsai 27B with right now and what variants of the weights are you using?