alt Hacker NewsIMHO lossy/lossless feels like the wrong term to use for natural-language text.
Every time natural-language passes from one human to another whether verbally or in the written word, there is a greater than zero percent chance that the meaning will be received/understood in a way that is "lossy" compared to the communicating party's intent. The so-called "telephone game" is a very good example of this.
It seems true that the lossy problem arises even without any LLMs or any intentional transformations.
Lossless in other domains basically means perfect-fidelity carried across copies. Natural language seems almost incapable of this ideal.
SHE said she did not take his money.
She SAID she did not take his money.
She said SHE did not take his money.
She said she DID not take his money.
She said she did NOT take his money.
She said she did not TAKE his money.
She said she did not take HIS money.
She said she did not take his MONEY.
8 very different messages all containing the same words in the same order. If the emphasis is changed or lost between sender and receiver, the message itself is lost regardless of the faithful reproduction of the actual words.