Neural nets are structured as matrix multiplication, yet, they are universal approximators.
You're missing the non-linear activations.
alt Hacker News
You're missing the non-linear activations.