Nice idea! JFYI, in case you ever run into design questions when extending this, you seem to be implementing a version of Object Capabilities (specifically revocable forwarders / caretaker pattern), specialized for the SMTP situation -- as you want to share with someone the capability to send you messages, but you want that capability to not be shared recursively.
Since you're also thinking about a user giving their agent access to their inbox -- any thoughts on what are the right capability primitives for that situation, and how the API & access tokens can represent that?
Agents can create new inboxes and inboxes can be of 3 types, besides "Permanent" which is just a regular inbox. The other types don't have "rules" that you can customize. Instead the rules are baked-in: Temporary -> expires after some time, One-sender: allows only one sender (the first one to prove its email identity), Prefix: similar to one-sender but creates a new inbox for every suffix (and if spammers catch on to the prefix, you can delete it and create a new one).