LLMs have helped me here the most. Adding copious detailed logging across the app on demand, then inspecting the logs to figure out the bug and even how to reproduce it.
I did that once: logging ended up taking 80% of the CPU leaving not enough overhead for everything else the system should do. Now I am more careful to figure out what is worth logging at all, and also to make sure disabled logs are quickly bypassed.
I did that once: logging ended up taking 80% of the CPU leaving not enough overhead for everything else the system should do. Now I am more careful to figure out what is worth logging at all, and also to make sure disabled logs are quickly bypassed.