Getting out of the way: my robotics crash course

I thought this was going to be a post about learning robotics.

It seems more like a post about hooking up an LLM to a pre-made robotic arm.

While that's interesting, I wouldn't have labeled the post "my robotics crash course."

Unfortunately, I think this is in some sense another example of LLMs substituting for actual learning. While I'm sure the author is learning something about robotics from doing these experiments, I doubt it's as much as he would have gotten from say reading a couple chapters of an introductory robotics for dummies book.

First law of calibration: Make sure you do an error analysis.

Finding the table surface is pretty useless using a top-down view, even with April tags, because the range error to an april tag is much more than the bearing (pixel) error to the april tag. You basically have trouble observing the thing you're trying to measure.

If you do this again, ask your agent to conduct this analysis and make sure your desired calibration variables are observable with small error. A second camera from a 45deg angle or even on the table would go a lot further, but then of course other things become unobservable.

Nice workaround using a proxy for force sensing to get touch info, however. And neat project overall!

Shoutout to people who don't overthink blog setups. "thisismypersonalblog.com" AND GO. It'll do the job.

On a technical note, your setup is not far from where professional setups are headed [1]. Astra is really doing an end-run around (for now) research Robotics setups.

[1]: https://x.com/ihorbeaver/status/2104646736652447854?s=20

Curious to see how they get on with VLAs. They sound great till you have to sit and record hundreds of teleop demos to teach them how to solve your task...

Anyone tried using Jev for fast AI robot decisions? Would it make sense?