An aside that I do want to mention here because it is a really unique way for many people to interface with LLMs: many commenters mention the model over indexing on a few comments they made that do not necessarily reflect of the broader themes of their writing. This is not any issue in the author’s engineering but an inherent issue in LLMs. The reason it is so noticeable in this case is because the subject matter is extremely familiar to the user: themselves.

LLMs consistently misrepresent information in this exact same way in, more critical applications. Because they are often employed on datasets that engineers and potentially end users are not deeply familiar with, the results often seem exceptional.

Disclaimer via my HN wrapped: “The Anti LLM Manifesto You will write a 5,000-word blog post on why a single Bayesian prior is more 'sentient' than GPT-6, and it will be ignored because the summary was generated by a 3B parameter model.”