Both Apple and Google are building the infrastructure for on device agents. I'm not as familiar with Apple's approach, but I've been hands-on with android AppFunctions. If you're familiar with Android ContentProviders and bound services, you've seen how apps can be custodians of the data they acquire and use.

AppFunctions enable tool calling with descriptions that are legible to LLM based This puts the apps in control of what agents can access, which is something they already mostly do.