What you're describing is similar to how the copilot harness in vs code tracks state and previous work. These systems are being implemented, bit by bit.