People building these Rube Goldberg contraptions: Do you actually run evaluations if this is any better at all than eg. giving it access to a Python REPL, or just toughing it out with random tools composed via shell scripts?

Why would an LLM be better trained to access Redis via a FS vs. a native library API?

Makes no sense.