logoalt Hacker News

libraryofbabellast Thursday at 1:11 AM1 replyview on HN

Oh yes! That's probably more important, in fact.


Replies

mnickylast Thursday at 7:35 AM

Well, I think that this is also answer to your question about the intuition.

If the assymetry of K and Q stems from the direction of the softmax application, it must also be the reason for the names of the matrices :)

And if you think about it, it makes sense that for each Key, weights to all of the Queries sum to 1 and not vice versa.

So this is my only intuition for the K and Q names.

(It may or may not be similar to the whole "db lookup thing"... I just don't use that one.)