alt Hacker NewsThe fact that by hand is emphasized so often and so often (see src/handgpt.h comments as well above backward()) makes me think it's AI.
Anyways, it looks like gradients derived by hand means they didn't use autograd. They have written out the expression for dL/dW themselves.
Replies
4gotunameagain • today at 9:33 AM
I think the readme is clearly LLM generated. The tone, the short sentences, the em dashes, lists, sections.. It all feels like LLM.
Correct. No autograd: the expressions for the gradients are written out explicitly in C. There's a gradient check in tests/ that verifies them against centred finite differences on the published header, worst relative error 1.07e-08.