Yes, agree with the general idea: permissions are fine-grained and adaptive based on what the agent has done.

IFC + object-capabilities are the natural generalization of exactly what you're describing.