Does anybody find it mildly ironic that LLM prompts, which are intended to be plain informal text, accumulate more and more structure around them, including a markup language in question?

This is not unlike the way the language of legal documents is highly formulaic, structured, and codified. When precise meaning is desirable, firmer structures tend to arise. With a bit more time, proper code languages may start to appear, to help tell LLMs exactly what we mean or want.