A LLM should at the very least be a language model, i.e. be able to take human-readable text as input or produce it as output. Transformers are used for plenty of tasks that don't involve language, for example object detection or blind source separation, where the models aren't called LMs; and on the other hand there are some LLM architectures that exclusively use linear attention variants and aren't really transformers anymore.