alt Hacker NewsIntroducing System One Models and Jev
Comments
I was thinking about something similar (maybe) - generally speaking, embeddings for LLMs tend to learn real world concepts - things like 'fruit' or 'France' or 'city' as directions in embeddings.
But in things like programming, most concepts are abstract - 'if hungry eat an apple' in programming terms would look like
'if hunger > 50 {apples--; hunger-=30;}'
and compilers work with 'concept erasure' - to them, tokens (which are like llm tokens) look like
'if var1 > 50 {var2--;var1-=30}'.
They don't care about how these things map to real concepts. So all the embedding directions used to encode real-world concepts are just noise to LLMs when programming. This greatly reduces dimensionality and training costs. So does a token representation tuned for programming constructs, rather than natural language would probably have a more efficient encoding.
We had early access and found it to be pretty useful. Having a second form of verification, where you can ask multiple questions (in the form of Nouls) raised our confidence in the outputs of other models. [0] IMHO This type of model works incredibly well in concert with LLMs, not as a replacement.
[0] https://goodstartlabs.com/research/verification-is-the-bottl...
This sounds good but so far all claims just sound like marketing terms. I'd love to see real proof. e.g. "RLCD" and "parallel sampling" have nothing to back it up.
also "70-500ms vs 3-329 seconds" are apples-to-oranges unless the LLM baseline is doing comparable work (e.g., long chain-of-thought). If Jev is skipping generation entirely for a narrow structured task, of course it's faster.
Nonetheless i want this to be true, so I'm looking forward to Jev
Edit: I really have to say that I like their manifesto https://typesafe.ai/manifesto
It seems like the docs[0] are a better explanation? The comparison to llm tokens is kinda confusing.
It looks like the model takes as input a state (structured text? not sure if multi-modal) and a question (as a "Choice", "Score", or "Noul") with some additional augmentations possible. Then outputs the question's answers as appropriate (e.g. a choice, accompanying probabilities, confidence).
Edit: On the AI primer page, it looks like they do the RLCD on a pre-trained base model?
I would love for things like this to be accessible via hubs like open router or AWS bedrock. It's hard to justify adding new model vendors directly with all the heightened concerns about privacy and security, but if bold new capabilities are added to a centralized already-vendor like AWS, technical people can adopt them without going through a whole compliance/purchasing/vendor review process. And an extra middleman tax is well worth it when the cost savings of the model itself can be one-two orders of magnitude.
Side note: it took me more time than I would like to admit to realize that Diogo Almeida isn’t a satirical version of the name Dario Amodei
From the person in the video regarding issues with benchmarks in general, and for LLMs. Also their approach. Good article.
I could see this being fantastic for classification tasks. Last year I shifted from using LLMs for bulk data classification tasks (1M transcripts) to generating embeddings and categorizing based on cosine similarity. It saved a ton of costs and time, but wasn't as accurate as LLMs. This seems like it can give me Terra-level classification ability with the cost/speed I need.
It could be used for coding if you gave it an AST.
If you work at TypeSafe please try this.
Side note: This is probably how LLMs would perform with better encoders and next-latent prediction, so eventually those will beat this architecture out. Still amazing though.
> Input tokens: $0.042 / MTok ($42 per billion tokens).
> Output tokens: FREE (too cheap to meter).
Insane. The video demos are really compelling, in particular the speed.
> Structured outputs slot into ordinary software as fuzzy decision rules: classify, route, score, extract, or branch where hand-written logic is too brittle. The surrounding code constrains their freedom, making them easier to compose into reliable systems.
I buy this vision. A lot of LLM integration I see these days is ultimately exactly this. OpenAI-style structured outputs works decently but this would be a great improvement in cost, latency.
Congrats on the launch! What's different between Jev and Microsoft's Guidance package? https://github.com/guidance-ai/guidance Is it a diffusion generator under the hood?
So in theory you could feed it incomplete text, and then ask it for the probabilities of what the next character could be?
https://x.com/completeskeptic/status/2099925682726002904?s=4...
The doom demo is quite cool
They never show exactly how they use it? Only a bunch of animations of it 'working'. Would like to see the actual code used for the demos!
After much fumbling around with prompts and evals, this is exactly how I am using LLMs in production, to narrowly make choices and return structured data. Any deterministic work gets pulled out of the prompt and my goal is to narrow the model output to be as clearly defined and as minimal as possible.
Jev's focus on structured I/O and confidence scores are game changing. If this does at all what it claims, I think this is going to quickly become the new standard approach for agentic systems.
> [others] Output tokens: ~5x more expensive than input tokens.
> [them] Output tokens: FREE (too cheap to meter).
I'm very confused by this.
Congrats on the release!
Finetuning a language model for decision classification (with probabilities) is already well-understood. What specifically changes in the training objective with RLCD? Are its benefits isolated from Jev’s new architecture/parallelism?
Interesting concept, I can't see a reason to use a generalist classifier over an api rather then just training my own? If it was open weights I would probably mess around with it.
Is there a downloadable technical report somewhere?
Will need hands on to truly tell, but the doom demo seems very promising. If it can play that with text descriptions of where stuff is by distance and degrees in a 3D context then many GUI automation tasks should be easily doable
If I’m understanding correctly, this will work well for self driving cars?
I’m not understanding what this is. It’s a faster cheaper LLM?
So is it a structured data-based language model? Or is there a model and a harness? Hopefully they’ll open up and explain more.
I am positive I know exactly how this works, I made something similar a few months back. But the problem is without generation you are extremely limited in the use cases. And while the model can't hallucinate, it can still be wrong. It just can't make up data.
What's the difference compared to just taking an embedding and feed forward a simple net trained for the task?
> We deliberately chose not to publish performance against public benchmarks. In fact, we plan to only have one-off evals when we make product updates.
lol, I bet they would publish them if their score on those benchmarks were good.
This puts the human even more out of the loop I'll guess?
Funny how it can do everything but not chat. Sort of how when I was a kid I thought of a medicine that could cure any disease except the common cold.
Is the tradeoff of the parallel output that we don't get arbitrary string generation? like output # of tokens is fixed ahead of time?
Either way, really cool and impressive.
Signed up for the beta! :) would love to put this through some real-world shootouts against traditional LLMs to see where this type of model really excels.
I’m guessing it might be able to replace maybe 40-70% of LLM calls for a given pipeline depending on the business task, cutting the API costs on those calls by an order of magnitude.
Super intrigued by this - large scale automation using LLMs is quite annoying due to deprecation cycles of models from frontier labs and cost of running your own being prohibitive when you have a blend of them.
Wild that it doesn't generate text. I wonder how its technology compares to Tesla's FSD stack.
Can this be used in practice to write code?
This sort of stuff almost sends shivers down my spine, it's like i'm looking 5 years into the future.
> Extraordinary claims require extraordinary evidence so see below for the receipts.
Yes, that’s the kind of attitude I want to see in these model releases
Why did they pick the name System One? It's not really explained what "System One tasks" and "System One shaped queries" are. Things that need a fast response?
Does this imply it's a very small model? I couldn't find anything about the model itself.
So... a classifier model?
This is basically a zero-shot classifier that can accept raw text (or structured text) as an input, and is able to classify that text as accurately (they claim) as a frontier-level LLM. I have workflows this would be useful for, looking forward to it showing up on OpenRouter.
Woof, that page is hard to read. I don't understand what they've done to the way text is rendering but it's not great for my eyes.
I could put this to use today.
I think we'll see a bunch of different architectures over the next five years.
Parallel inference where you don't want a subagent seems niche. But there is a lot of random things where businesses ultimately want some kind of score instead of generating something.
I think the interesting thing would be seeing if prompt injections still work with this kind of model.
um what is going on with the outfit changes in the launch video...
What is it about the rendering of this page that is so... off? It almost looks like the entire thing is a <canvas> element.
edit: looks like a framer export where there is a text stroke being applied :|
Looks like a great model for NLP.
this is interesting, so not an LLM but can be used in these use cases that LLM's have been shoehorned into
oooooh it can play Doom!
forget LLM benchmaxxing sidequests, I'm sold on the real benchmark
"is this the real thing or is just fantasy"
First, congrats to the team on launching something genuinely interesting and new.
Seems like a more accurate title would be "Jev: Trading general purpose generation for fast typed inference" or something like that.
This is interesting, but the speed comparison seems misleading? A generative model that can output code in a Turing-complete language can do anything a computer can do.
Jev can only generate structured output, right? This is probably super useful for classification/routing/scoring, but it's nothing like the code generating models we're all using today for code and automation.
Also "can't hallucinate" seems wrong? Sure, it can't emit an invalid type, but it can still emit a completely wrong valid value. You can enforce structured output from an LLM too, with an appropriate harness, etc.
Assuming there's no funny business, the Doom demo is cool.