alt Hacker News> "post-trained to know when it's wrong"
Is it also post-trained to know when it's wrong about when it's wrong?
> "Every response comes with a confidence score between 0 and 1"
How confident is it in its confidence?
Please, I'm sure that what you're doing is very neat and useful, but use other language to describe it. I beg you. You can't know when you're wrong. You can only know when you're unsure or inconsistent. You can be absolutely certain and still wrong and uncertain and still correct.
IMHO this also depends on the type of assertion.. For example, one system defines three broad classes of assertion : pure opinion (I like green); factual references (the current temperature here is 20c) ; and the rest in a catch-all about "reasoned judgements"
Complex assertions and their compounded forms can be filtered by these classes fairly easily. I rarely see this referenced in general comments.