logoalt Hacker News

dingaling911 • today at 2:31 PM • 2 replies • view on HN

DNA is probably the seed for the random initialization.


Replies

nico • today at 3:05 PM

This is actually a genius idea. Imagine being able to compress a 1T parameter model to 10M params by creating a lossy model-scaffolding, like DNA

Something that more or less guarantees you’ll get about the same capabilities as the original model, but with a different final expression of certain characteristics

I wonder if this could be a good distribution model too, selling “dried” models (almost like raisins - or the beings from the 3 body problem), and then you can buy them to fully train (raise?) with your own data

➕ show 1 reply
wernam • today at 2:42 PM

It's nice to throw around claims like that, or like "microtubules connect the brain the quantum realm" (Penrose). Fortunately that's not how science works.

➕ show 2 replies