alt Hacker NewsIf “scale will solve everything”, even (as the article contends) things that could be solved more cheaply in other ways, that’s of course wasteful and inefficient.
But what about things that only scale can achieve? Like the superhuman security vulnerability assessment capabilities that Fable showed? That would be a reason to continue to spend, wouldn’t it?
Replies
B1FF_PSUVM • last Saturday at 12:44 PM
> scale will solve
I have a bad feeling about this, and it's about us, not AIs ...
(I fear that we're #$&@%!## most of the time, and just oblivious about it)
Do we know that Fable/Mythos was the result of throwing more hardware & data at it? Anthropic is still pretty compute constrained. More does not always equal better, it very well could have been Fable/Mythos came as a result of better data curation or some other break through, not necessary more parameters & compute.
I don't think "just throw more compute at it forever" is the only way to go, but if that turns out to be true, the labs aren't going to share that knowledge because that would be a risk to the dump trucks of cash getting dumped at their feet if they came out and said "You know, we don't really need much more compute, we found a better way to make a smarter model" the cash would slow down.