alt Hacker NewsOllaya – Ollama for open-source, Jev-style decision models
Comments
I'm not sure what this means for AI startups if their innovations can be copied by OSS so quickly (what, like 2 weeks?). There's "consumer surplus" for everyone, to borrow an economic concept. But we do ideally want some of the surplus to flow to the innovator, too. I know there were precursors, but that's fine - it's hard to have a totally novel idea in such a popular field. I don't know what the end game is for TypeSafe - they'd need to demonstrate perpetually better results, or compete in another axis: UX, support, custom solutions, etc. So much of the time, someone proving a concept, or it simply getting enough publicity, is enough for a "Cambrian explosion" of follow-ups and copies. Famously, that was true for "Attention is All You Need", and the general idea of "next-token prediction" being so powerful.
We've stumbled into general differentiable models..
Guys I have a real q, what is the difference between an instruct based re-ranker and laya/jev I just don't see it.
Edit: One is that jev/laya are tuned to have better probabilities, but a reranker can be fine tuned to do that as well. And jev/laya use RLCD?
Has anyone actually seen better or the same results with Laya compared to Jev? From my experience, Laya performs significantly worse. It's less confident and often makes wrong decisions with more complex queries.
>Run decision models locally.
>example is a text classification task instead of a decision
Would be great if you supported CUDA 12; I don't feel like paying $15K to upgrade my GPU right now
FAQ[1] says:
> It is an independent project, not affiliated with Ollama.
Sounds good on latency but how is its actual decision quality vs. Jev?
It would be really cool to have LLMs and System One in a single tool - in this case, if Ollama implemented it.
Does anyone know what laya multi lang is faster than laya en? I would have thought focusing on a single language would be faster.
Are there many models that are comparable to Jev for generic decision making?
Smarter move if you have an eval set is to just train a classifier and call it a day.
Cool... but this does seem undermined by the fact that Ollama can add support for decision models at any time.
I have also tried this and its really awesome
I am fairly confident if Jev-style decision models are seen as prominent (which, they seem to be), Ollama will support them. Surprised the team hasn't implemented this already.
great project for empowering open-source alternatives.
[dead]
Hey Claude, make ollama for Jev like models. Make no mistakes /s
This inference engine is soooo much faster btw: https://github.com/tamnd/kime
Laya is pretty easy to set up on its own without ollaya. I just did that and replaced my current jev API usage to laya running on a GTX 970 with 4GB of vram.
Very small context window, but for some existing small llm work I was doing, it was a drop-in replacement and it makes me happy I can get use out of old hardware I have running.