logoalt Hacker News

crystal_revengetoday at 3:40 AM1 replyview on HN

> There’s an uncensored model floating around that you can run locally with llama.cpp

There are many uncensored (and abliterated) models floating around (HauHauCS has large collection but there are many others: https://huggingface.co/HauhauCS). I'm using `Qwen3.6-35B-A3B-Uncensored-Q4_K_M` (the one referenced in your link) because I find it's writing style much more interesting when you push go off the guardrails a bit, and because I think self-censoring when effectively using an advanced journal is variety of dystopian I'm not ready to accept

> it’s annoying to use since you run out of context window quickly, and it’s certainly not able to be deployed in production (i.e. Tom Riddle’s diary as a service).

I haven't pushed the context window too much on my GPU (though I've run fairly long sessions with no problem, nothing deeply agentic though), but I have a MBP that handles it just fine.

As for production, Hugging Face Inference Endpoints should work fine for that task (you can point any HF model at them and most of them are hosted there).

> For better or worse, fun is no longer allowed.

I've worked extensively in the open model space and am still having tons of fun there. If anything it's gotten aggressively better in recent months.


Replies

sillysaurusxtoday at 3:46 AM

Thanks for telling me about Inference Endpoints. That’s awesome.

So glad local models are getting good enough to be deployed. The uncensored model’s output was far better than expected in a domain that triggers guardrails with ChatGPT and Claude.