Design work with AI is definitely a game of balancing creativity and flexibility with the constraints defined in the design system, and it's been something we've thought a lot about.
Onlook lets you add browsers and images to your projects to be used as context, so it's already a strong tool for working with design references
Another way we're addressing this is by keeping the projects you're working on separate from the self contained design artifacts you may generate in the process. Since these can be easily used across projects we find that it's very low friction to take a cool but discarded thing you sketched once and repurpose it for another project where it may fit in better.