> While deterministic, there is not a fixed number of ways it can express itself, given that we can use settings like temperature to inject randomness into the output.

You're missing the point, which is that no matter the process involved. The LLM can only ever output one of the tokens in its token vector. It can't invent a new symbol or character. It can't leave and go build a church. It has to output a little piece of data for you.