alt Hacker NewsBut they don't. That's the whole reason they work in the first place.
Every neuron in the layer encodes fractionally something about the concepts embedded in the whole thing. It's more like a hologram than a series of connected facts.
But they don't. That's the whole reason they work in the first place.
Every neuron in the layer encodes fractionally something about the concepts embedded in the whole thing. It's more like a hologram than a series of connected facts.
Nick's claim is that that's the fundamental misinterpretation. Each neuron eventually ends up mostly doing one main thing (and the rare case that some neurons do several related things). Neurons compose to embed ideas which is a starkly different interpretation than them being holograms and fractional encodings.