logoalt Hacker News

machina_ex_deus • today at 9:24 AM • 0 replies • view on HN

Humans do something even "worse" - all their "weight matrices" e.g. neurons activation patterns are constantly changing while they think. If every llm had to have it's weight matrix updated during inference, that would be orders of magnitude more costly than merely having n^2 attention.