I disagree with each sentence for a different reason.

LLMs interpret (so, "execute" in a way) natural language in the sense they have internal logic that assigns to the sequence of tokens in context a next token. If we delineate the input and output into a series of logical statements, we can think of it as a program that builds a logical statement from a list of input statements. So it encodes derivation in some logical system.

However, the internal logical system is informal in the sense that the above rules are not guaranteed to be sound on the fragment of classical logic encoded in the natural language. It is a close approximation, though, so it often works.

To add, half of my problem with natural language would be resolved by agreeing on exact definitions, which is kinda what LLMs do internally. However, they don't surface this formalization very well(even with open weights it's difficult), which makes it pretty unusable.