logoalt Hacker News

h8hawktoday at 7:56 AM1 replyview on HN

Hinton did not invent backpropagation.

related paragraph from Wikipedia:

Modern backpropagation was first published by Seppo Linnainmaa as "reverse mode of automatic differentiation" (1970)[26] for discrete connected networks of nested differentiable functions.[27][28][29]

In 1982, Paul Werbos applied backpropagation to MLPs in the way that has become standard.


Replies

ogriseltoday at 9:33 AM

Paul Werbos did not apply backprop to MLPs as cleanly described in Hinton's paper, but rather to some kind of autoregressive non-linear parametrized functions with a much more specific application scope.

Both papers are direct applications of the chain rule applied to estimate the gradient of a multivariate function.