The Open/Closed Problem in AI

Storing configuration and “memory” outside the LLM in prompts and skills written in English and in tools written in ordinary programming languages makes them far easier to modify by ordinary programmers who aren’t AI experts, and we can use the coding skills of LLM’s to modify them too.

It seems a lot more “open” and accessible than it would to store memories in an opaque neuralese that we have to do mechanistic interpretability research on to get even a partial understanding.

There’s a lot of churn in the agent harness space but I expect to see innovation come out of it, too.

This kind of continuous fine tuning is being actively researched very heavily, but you might not see much about it at a conference because the labs don’t have enough to show their cards yet.

I assume there's just too much money in optimizing for the thing that works now and making it faster than in risking it all on unproven approaches.