This is completely expected. Agents are pseudo-greedy search functions that brute-force solutions using their context.
If you want perfect search, you need lossless compression, which defeats the point of the current systems that are lossy L2-optimized representations of all information.
[dead]