alt Hacker NewsI'm biased but I wouldn't call it misleading - generating text is super awesome and flexible, (we describe that in the blog post - and I personally use string models all the time) but it's true you pay a high tax for autoregressive generation
> Also "can't hallucinate" seems wrong? Sure, it can't emit an invalid type, but it can still emit a completely wrong valid value.
that is likely true of all ML! perhaps we could debate semantics, but I don't think it's fair to say a random forest "hallucinates" in the way LLMs do
Replies
His claim was that the title is misleading, not sure how it's relevant to that claim that you use "string models" (full LLMs).
The original title before it changed less than an hour ago was:
"Jev: New frontier model 40-400x cheaper and 20-200x faster"
I'm going to agree that was misleading.
And on the second point:
>>Also "can't hallucinate" seems wrong? Sure, it can't emit an invalid type, but it can still emit a completely wrong valid value.
>that is likely true of all ML! perhaps we could debate semantics, but I don't think it's fair to say a random forest "hallucinates" in the way LLMs do"
Also going to disagree here, and I don't think it's semantics.
Type safety is not factual correctness.
From a quick look at this it looks like it could easily generate natural language text by following a structured representation like UMR (Uniform Meaning Representation) or the similar representation the Abstract-Wikipedia folks will be working on for generic encyclopedic text (which will be heavily informed by Universal Dependencies). These are basically linguistically principled and frame-based counterparts to a programming language AST, that can be then converted to natural language (in a broadly language-independent way, to the extent that semantics and pragmatics make that feasible) via some sort of NLG rendering.
(To be clear, this one raw model does not support outputing a full AST directly - it wants to output "choice" among fixed options, "score" on a sliding scale, or a true/false answer (all of these with confidence scores attached), so building the AST/structure would be a code-driven (or even perhaps outside LLM-driven in some more challenging cases) multi-step affair where the model would essentially be playing a "game" of building the structured output step by step and getting a revised partial state back. But one could expect this to lead to interesting results.)
> I'm biased but I wouldn't call it misleading
- @CompleteSkeptic
Very strange.
Just to be sure that I understand, you're saying that your model "can't hallucinate" because it only outputs a single thing, right? In this way, an LLM can't hallucinate either if I prompt it to do a classification task with a discrete set of possible outputs, right? (Assuming I reject non-conforming output. Actually, maybe what you're saying is that your system can't output non-conforming output?)