If anyone from cloudflare comes here - it's not possible to create D1 databases on the fly and interact them because databases must be mentioned in the worker bindings.
This hampers the per user databases workflow.
Would be awesome if a fix lands.
(I work at Cloudflare, but not on D1)
I believe this is possible, you can create D1 databases[1] using Cloudflare's APIs and then deploy a worker using the API as well[2].
1 - https://developers.cloudflare.com/api/resources/d1/subresour...
2 - https://developers.cloudflare.com/api/resources/workers/subr...
I'm always a little hesitant to use D1 due to some of these constraints. I know I may not ever hit 10GB for some of my side projects so I just neglect sharding, but also it unsettles me that it's a hard cap.
Why not durable objects? I think it's the recommended pattern for having a db per user
Try Durable Objects. D1 is actually just a thin layer over Durable Objects. In the past D1 provided a lot of DX benefits like better observability, but those are increasingly being merged back into DO directly.
What is a Durable Object? It's just a Worker that has a name, so you can route messages specifically to it from other Workers. Each one also has its own SQLite database attached. In fact, the SQLite database is local, so you can query it synchronously (no awaits), which makes a lot of stuff faster and easier. You can easily create millions of Durable Objects.
(I am the lead engineer for Workers.)