> the logic being you don't want to do that every time for every prompt

That's definitely a downside of a minimal MD and I could def be wasting tokens. I've tried to scope the tasks such that it won't feel the need to scan unnecessary files, but it's not bulletproof.