It’s not a 3B model, it has 3B active parameters. The full model is much larger.
That's true, I should have mentioned active. Actual params are closer to 12B-14B likely, given the 40GB VRAM usage.
alt Hacker News
That's true, I should have mentioned active. Actual params are closer to 12B-14B likely, given the 40GB VRAM usage.