alt Hacker Newsthe 7b distilled version works great on my laptop cpu and iGPU with vulkan. You can use llama.cpp (for iGPU with vulkan) or ollama (for cpu).
My laptop is a cheap laptop from 5 years ago. Not cutting edge hardware.
the 7b distilled version works great on my laptop cpu and iGPU with vulkan. You can use llama.cpp (for iGPU with vulkan) or ollama (for cpu).
My laptop is a cheap laptop from 5 years ago. Not cutting edge hardware.