alt Hacker NewsJev in 25 Lines of Python
Comments
I am hearing about Jev for the first time here so no idea about the hype. So their(Jev) is that the thing is faster at classification than a frontier model? Because the whole type safe aspect is already fully solvable with structured output. But their example is classification but that would also be possible and faster with a classic BERT model. So their pitch is a task specific smaller model or am I completely misunderstanding the whole thing?
Because of masked attention in LLMs, if you put the options before the body (the email to analyze), the transformer already knows what it needs to look for, and can use more tokens to create state to address that specific task (BERT has no mask in the attention, so tokens attend also to next tokens). You could also do a few examples in the system prompt to improve calibration.
Another trick that works is to repeat the question two times: "I'm repeating the task and labels for clarity: ..."
Beyond the missing latency and compute comparisons that Heaney commenter mentioned, also nothing about its error rate compared to Jev (nor if it even always outputs in a format the app can parse, not sure how solved that is).
But then at the end it says it’s parody. Maybe HN title should say it’s a joke.
Looking at the logprobs on tokens works for the local models, but not on the frontier ones. It's been more or less broken since GPT-4o for example. I wrote about it two years ago: https://medium.com/data-science/9-11-or-9-9-which-one-is-hig.... Also, I've done some work in estimating confidence and on rubric evals using the same method, and you actually get better correlation to "real confidence" by just getting the LLM to say it.
Whilst I do like reading these things for technical know how, I can sympathise with the creator of jev who now presumably has to apply an order of magnitude effort to explain why the 100 smaller things done better than this add up to a much better product.
Highly suspect of content marketing.
Ends with referring to a product, and saying "this is a parody post", after pretending to make a serious point.
How to write Jev in 25 lines of Python:
1. draw a circle
2. import the rest of the owlIt's fast.
If you're comparing with something, you need to state 'fast' in relative terms. Jev is definitely fast, and if this Python takes the same time to get a decision then it's also fast. If it's 100* slower than Jev though, you shouldn't be calling it 'fast', because relatively speaking it's really, really slow.
We have Jev at home
The one thing I can't wrap my head around with Jev is why they're trying to create that "System One" narrative.
In real life, a human doesn't do classification tasks with the System One part of their brain, they use System Two. So by definition what Jev does isn't System One thinking.
If anything, regular programming that automatically executes based on logic, without requiring "thinking" would be "System One".
I missed the hypewave so can't say a lot about Jev, but the double standards are entertaining:
About Jev:
> We didn't train a model with Reinforcement Learning for Calibrated Decisions (RLCD) to calibrate the decisions and probabilities (even though they are not always correct).
Only 99% correctness! Borderline unusable!
About their model:
> It classifies: it gets a prompt with choices and outputs probabilities.
You want numbers, it gives you numbers! What more could you want?
I wonder if this could be a good stepping stone to write a local prompt router to optimise what model get what prompt. I.e. if the prompt is just a lookup, send it to haiku, if it's reasoning, send it to opus and if it's implementation send it to sonnet.
a hile ago (when big providers still provided logprobs) i created a VS Code highlighter that visualizes unsure tokens.
Since most chat models want to answer with a human-readable message i think their logprobs are not as meaningful. It would be interesting to see if one choice is like "correct" and if the model wants to choose it more often, cause it might not answer the question but to prose to the user.
What I'm missing here is also type guarantees. I don't think you can do it without token level logic which forces the model to output the tokens from a predefined pool of tokens. A logic like this given some JSON schema is not that difficult to implement. If the LLM must output JSON schema compatible value then you can also add that it doesn't "hallucinate". Which is funny too because just guaranteeing the type does not mean the model does not hallucinate but this is another story.
I'm surprised something like Jev came out "so late", but the hype has been ridiculous. Yes, it's a good idea. No, it only helps when fast and cheap are important and I guarantee existing labs will have this figured out in a matter of days.
Add visual understanding, add reasoning and bring down the size to run on my computer. That's when it will be interesting.
So many people that don't understand the tech jumped on the hype train because "it cannot hallucinate" and else. It's crazy.
Reminds me of my own tiny Lisp interpreter attempts; that moment when it first evaluates a simple expression is pure magic.
You can also go beyond Jev. Qwen 3.5 0.8B is fantastic at basic image classification/question answering (including OCR elements) also. Though rather than looking at logits, I get it to output a structured JSON object and it does simple object classification tasks on a Mac at under 500ms a pop (I forget how far, but I think it's like ~250ms) with good accuracy (depending on task).
I've built something similar, i am hosting it.
You can test Jev like model at 26B parameter count here (built few weeks ago): https://gambler-relay-us-west1.leo-fish.ts.net/demo (might not stay up for long)
Typesafe compatible API
This is just running on old hardware.
strong "You can build dropbox quite trivially by getting an FTP account, mounting it locally with curlftpfs, and then using SVN or CVS on the mounted filesystem" vibes
You have built something like jev but not jev (for starters, the output of what you've built will be absolutely worthless, the whole reason Jev is getting so much hype is because the output is good enough)
Pretty interesting how a simple example like this makes the idea so easy to understand.
What I don’t understand is, why would you not want “reasoning” in a classifier?
Speed and cost are obvious reasons, but isn’t this a tradeoff?
Startup coming out of 2 years of stealth to be reproduced this easily
Now, can you do it in <200ms for 45 questions at once, have 0% malformed output, and any kind of meaningful benchmark? We’ll wait!
I get dishonest vibes from this post? Jev claims to be cheaper/more efficient, and the post claims just to achieve the same functionality.
The principle is this.
I'm so sick of seeing these people who "made Jev in 25 lines of Python" or whatever the flavor of the day is. Do you people seriously think that Qwen3-0.6B-Q8_0.gguf is frontier intelligence? If you want to argue that Jev is NOT frontier intelligence, then go make that argument. Don't try to pretend that Qwen3-0.6B-Q8_0.gguf is frontier intelligence. That's retarded.
Nothing I hate more than bullshit articles claiming X in Y lines of code, only to use libraries abstracting hundreds of thousands of lines of code.
[dead]
[dead]
[dead]
[dead]
Going directly for the logprobs is always icky when you use a chat model as base, because they are trained to write prose as output. So your "choice" tokens and thus their probabilities might get diluted in whatever else it wanted to say. If you have to do it in the same way as this post, at least add clear system instructions and a carefully worded beginning to the assistant output section of the prompt to lower the chances of it wandering off immediately.
I've found that using structured outputs solves this problem much better. Instead of letting a model generate only "A", "B" or "C" and looking at the probs, have it directly generate "Legitimate", "Spam" or "Phishing" or any other pre-defined option from a set of multi-token sequences. Behind the scenes it boils down to something quite similar, but you're not running into the risk that the model actually wanted to say "A phishing attempt seems likely, so answer (C) is correct.", which would lead "A" to have the highest probability in the first token. You can even use a reasoning budget this way either via inherent reasoning or a free-form part preceding the remaining output structure. You can also have it assign probabilities (either in words or numbers) using more complex output structures, but I would not rely on them much more than the token logprobs (they can still be quite good though).