logoalt Hacker News

jumploopsyesterday at 11:35 PM1 replyview on HN

That's actually how it started, but with my own opinionated skills[0].

One thing I discovered was that the worker agent, having access to all the skills, would sometimes expand scope unnecessarily.

This led to the agent making the solution "better" than the initial request, which is what I want most of the time in my actual development (e.g. /tmp/frame-N.bmp instead of a single /tmp/frame.bmp).

I ended up testing a flow where the supervisor chooses the skill(s), and only injects the subset into the worker. Not sure I love it, but it made the worker execution cleaner.

For the verifier (not documented in the blog post), I used a fresh-worker context that would attempt to adversarially poke holes in the solution. This worked pretty well, but required increasing the timeout by 2-3x (thus invalidating the benchmark).

[0]https://github.com/jumploops/chum


Replies

mtzaldoyesterday at 11:41 PM

Yes! That's a great solution. I mostly use tdd, and code coverage and a validator afterwards. Skills are of a great way to guide the agent and context too.

Once the specs are being completed and splitted into beads, I span multiple agents (ultreworkers) and as part of a contributing guidelines I specify to use gitflow + git worktrees, then pr.