logoalt Hacker News

xyzsparetimexyzlast Sunday at 1:39 PM5 repliesview on HN

Couldn't someone build an adaptive system, where the llm is frequently judging the difficulty of a task and switching to a more/less powerful model?


Replies

iamflimflam1last Sunday at 1:55 PM

This only works well if you are creating sub agents with clean contexts for each task. If you constantly are switching models part way through some work then the whole session needs to be replayed each switch. You lose all the benefits of the context cache.

show 1 reply
bob1029last Sunday at 2:55 PM

The judgment of task difficulty is a cursed problem. In most domains you really don't know what is down there until you start digging a little bit.

show 1 reply
bkotryslast Sunday at 1:55 PM

definitely something that will come sooner than later. In my view, adaption of model routing / model orchestration and mixing providers is just natural next step.

zombotyesterday at 5:06 AM

And while you're at it, solve the halting problem.