alt Hacker NewsThis has to be the least surprising development to date given ml is a universal function estimator
Replies
This is probably the most important concept for "normies" to understand about AI, IMO. It's the stochastic brother of the deterministic Church-Turing thesis. Any function that can be computed can be computed on any computer. And that function can be approximated to an arbitrary degree of precision with a DNN.
The real kicker is DNNs are much easier to program than CPUs because they don't require a closed-form description ("a program") of the function to be approximated; you just throw a bunch of input/output pairs at the model, compute loss, backprop and update weights, repeat.
Hence the unslakeable thirst for input/output pairs, i.e. data.
> In the field of machine learning, the universal approximation theorems (UATs) state that
> neural networks with a certain structure can, in principle, approximate any continuous
> function to any desired degree of accuracy. These theorems provide a mathematical
> justification for using neural networks, assuring researchers that a sufficiently large or
> deep network can model the complex, non-linear relationships often found in real-world data.[1][2]
>
> The best-known version of the theorem applies to feedforward networks with a single hidden
> layer. It states that if the layer's activation function is non-polynomial (which is true
> for common choices like the sigmoid function or ReLU), then the network can act as a
> "universal approximator." Universality is achieved by increasing the number of neurons in
> the hidden layer, making the network "wider." Other versions of the theorem show that
> universality can also be achieved by keeping the network's width fixed but increasing its
> number of layers, making it "deeper."
https://en.wikipedia.org/wiki/Universal_approximation_theore...
Not sure that is enough for forecasting as the function to be estimated could change over time in random ways.
You mean neural networks?
As someone who started working on AI forecasting 3 years ago, I can confidently say that most people did not expect AI to beat Tetlock's superforecasters, Metaculus pros, or prediction markets as quickly as it did.