alt Hacker News> And if you ask an LLM about an effective cure for cancer and it spits out to drink bleach, are you going to follow it?
But it's not going to do that. It's literally designed to give the best and most accurate answer that it can. It's not perfect and I don't expect it to be perfect. According to my personal experience it's definitely good enough for a lot of things like recipes, coding, hobby stuff, and home repairs.
You're expecting me not to trust my own personal learned experience using AI as if somehow all my continued successes are worth nothing because you can invent some wild hypothetical. The more I use it, the more I understand its limitations and avoid them.
Replies
> But it's not going to do that. It's literally designed to give the best and most accurate answer that it can.
No, no, no and no. This is the biggest mistake I see people consistently make with LLMs. It is not designed to give you the most accurate answer, it is designed to give you the most likely series of words following your prompt.
If an LLM is trained on 10 jackasses thinking bleach is a medicinal drink and 1 doctor who disagrees, it will by virtue of probability tell you to drink bleach. Companies add additional safeguards or system prompts to try and keep it 'on rails' but it's all probability based on what you prompt it. It is by the literal functionality of how it works to do so. A function depending on what said company ingests during training, many of which include the entire corpus of the internet.
If you do not understand how LLMs work under the hood then yes, you shouldn't trust your own personal learned experience because you've already demonstrated that you're wrong.
> But it's not going to do that. It's literally designed to give the best and most accurate answer that it can.
This is not true. This is not how LLM's work. They have no concept of accuracy or "best".