Reading that made me think how much that might be related to Elixir being very similar in syntax to Ruby. Do LLMs really differentiate between the two?

Specific studies, as the one quoted, are a long way from original real world problems.