logoalt Hacker News

CGMthrowawaylast Monday at 1:41 AM2 repliesview on HN

That makes sense. Why does least squares skew the line downwards though (Vs some other direction)? Seems arbitrary


Replies

mr_toadlast Monday at 9:24 AM

The Pythagorean distance would assume that some of the distance (difference) is on the x axis, and some on the y axis, and the total distance is orthogonal to the fitted line.

OLS assumes that x is given, and the distance is entirely due to the variance in y, (so parallel to the y axis). It’s not the line that’s skewed, it’s the space.

sega_sailast Monday at 2:37 AM

I think it has to do with the ratio of \Sigma_xx, \Sigma_yy. I don't have time to verify that, but it should be easy to check analytically.