alt Hacker NewsDoesn't this indicate that there's not a lot of transfer learning between languages? If the networks read a prompt, reason about it and then present the result, only the reading and writing part would differ between languages, and that should be a small part of the full energy use.
It would be interesting to see if it would make any difference if you posttrained with translations of the same texts, i.e. focusing on teaching the model to specifically translate concepts between languages.
Replies
"Reason about it" is where your abstraction has sprung a leak.
LLMs do what they say on the tin: they model language. If you ask an LLM "find security vulnerabilities in my code" for hello_world.rs, the next likely tokens are "I found the following vulnerabilities", regardless of what logic or reason say.
It happens to be true that predicting next tokens on your own output mocks actual reasoning fairly well. But it still just a simulacrum rather than the real thing, and when held as an abstraction it leaks everywhere.
Understanding this is critical to avoiding the common failure case of scope creep and code slop.
My understanding is that it’s about tokenization. English input is converted into less tokens because tokenizer is optimized for English. This makes huge difference because all the output tokens are fed back into the input at each token generation. And attention blocks are generally quadratic with the input tokens. So foreign language means more tokens everywhere and more tokens mean more passes and much more compute and memory.