So if I understand correctly, in an agent, the LLM is in charge, but it can send part of the work off to other tools. And the problem here is that we're trying to have something in charge over the LLM, which is the reverse of the "agent" setup. Do I have that right?
Yeah, OpenClaw agents have a full set of tools to interact with a browser in arbitrary ways. My idea was to instead give it a tool for a browser wrapper with a limited API surface. And that tool could use LLMs internally in specific contexts.