alt Hacker NewsI often see things like this and get a little bit of FOMO because I'd love to see what I can get out of this but I'm just not willing to upload all these private documents of mine to other people's computers where they're likely to be stored for training or advertising purposes.
How are you guys dealing with this risk? I'm sure on this site nobody is naive to the potential harms of tech, but if you're able to articulate how you've figured out that the risk is worth the benefits to you I'd love to hear it. I don't think I'm being to cynical to wait for either local LLMs to get good or for me to be able to afford expensive GPUs for current local LLMs, but maybe I should be time-discounting a bit harder?
I'm happy to elaborate on why I find it dangerous, too, if this is too vague. Just really would like to have a more nuanced opinion here.
Replies
The docs I upload are ones I'd be OK getting leaked. That also includes code. Even more broadly, it also includes whatever pics I put onto social media, including chat groups like Telegram.
This does mean that, useful as e.g. Claude Code is, for any business with NDA-type obligations, I don't think I could recommend it over a locally hosted model, even though the machine needed to run a decent local model might cost €10k (with current price increases due to demand exceeding supply), that the machine is still slower than what hosts the hosted models, that the rapid rate of improvement means a 3-month delay between SOTA in open-weights and private-weights is enough to matter*.
But until then? If I'm vibe coding a video game I'd give away for free anyway, or copy-editing a blog post that's public anyway, or using it to help with some short stories that I'd never be able to charge money for, or uploading pictures of the plants in my garden right by the public road… that's fine.
* When the music (money for training) stops, it could be just about any provider whose model is best, whatever that is is likely to still get distilled down fairly cheaply and/or some 3-month-old open-weights model is likely to get fine-tuned for each task fairly cheaply; independently of this, without the hyper-scalers the supply chains may shift back from DCs to PCs and make local models much more affordable.
I've been analyzing my Obsidian vault using local LLMs that I run via Apple's mlx_lm. I'm on an M4 MacBook Pro with 48GB RAM.
The results are ... okay. The biggest problem is that I can't run some of the largest models on my hardware. The ones I'm running (mostly Qwen 3 at different numbers of parameters and quantization levels) often produce hallucinations. Overall, I can't say this is a practical or useful setup, but I'm just playing around so I don't mind.
That said, I doubt SOTA models would be that much better at this task. IMO LLM generated summaries and insights are never very good or useful. They're fine for assessing whether a particular text is worth reading, but they often extract the wrong information, or miss some critical information, or over-focus on one specific part of the text.
I don't really buy this post. LLMs are still pretty weak at long contexts and asking them to find some patterns in data usually leads to very superficial results.
It really depends on how deep you want to go. And this will likely not be useful in any way, other than a new hobby. Me and my friends who do this kind of thing, we do it for fun.
If it was not fun for me, I would not have bought 3 GPUs just to run better local LLMs. Actual time, effort and money spent on my local setup compared to the value I get does not justify it at all. For 99% of the things I do I could have just used an API and paid like $17 in total. Though it would not have been as fun. For the other 1% I could have just rented some machine in cloud and ran LLMs there.
If you don't have your private crypto keys in your notes worth millions, but still worry about your privacy, I'd recommend just renting a machine/GPU in a smaller cloud provider (not the big 3 or 5) and do these kind of things there.
OP's post is trying to con you into using a LLMs for a task that they do not perform well at.
This is specific, but if you start replying to LLM summaries of emails, instead of reading and responding to the content of the email itself, you are quickly going to become a burden socially.
The people you are responding to __will__ be able to tell, and will dislike you for your lack of consideration.
> I'm just not willing to upload all these private documents of mine to other people's computers where they're likely to be stored for training or advertising purposes.
And rightfully so. I've been looking at local LLMs because of that and they are slowly getting there. They will not be as "smart" as the big models, but even a 30B model (which you can easily run on a modern Macbook!) can do some summarization.
I just hope software for this will start getting better, because at the moment there is a plethora of apps, none of which are easy to use or even work with a larger number of documents.