Anthropic: Introducing The Conceptual Reasoning Index

Ah, yes -- A closed source benchmark that Anthropic paid for that Anthropic ranked highest.

0/10

Opening line:

> A core hope for managing AI risks is that AIs will help us understand our situation

Gonna stop you right there and ask that you think deeply about that premise.

We invented a new benchmark and look we're at the top. Everyone else sucks compared to us. Especially those dirty open models.

This is a modest start on an important direction for AI Alignment work; which is, as the authors observe, commonly comprised of tasks which are not readily empirically verifiable and not easily mathematically modeled - so it's hard to get at with normal RL techniques.

I find the ACCoRD benchmark the most interesting, because you could theoretically scale it up from the baseline mode of testing two instances of the same model for their `P(A) ≥ P(A&B)` respectively, you could do `P(A)≥ P(A&B) && P(A) ≥ P(A&C) && P(A&B) ≥ P(A&B&C) && P(A&C) ≥ P(A&B&C) ...` etc

i.e. a swarm of model instances could be collectively measured for consistency for even more confidence, right?

At any rate, even the basic idea of measuring a model for consistency in beliefs improves our ability to bound the amount of trust we can put on it with introspection methods.

> For example, if we ask a model for the probability P(A) and another instance of the same model for the probability P(A&B), do the reported probabilities satisfy P(A) ≥ P(A&B)?

So that's it? Are you saying they compressed risk reduction to high school level stats calculations and a greater than or equal to? To early for this

I think this says as much as the leader on the index as it says about the loser. If I believe I do not want to trust the AI to know best what is it that I want done, then this tells me not to go with Anthropic.

I think we can mostly eyeball it at this point. There hasn't been a model that I've thrown a novel problem at that didn't turn into an iterative token bonfire until I intervened and until that has changed, most of these benchmarks feel kind of like pointless marketing slop.

I don't remember any company in world's history that has both been loved and hated by the same users who purchase from it. We love Anthropic for its amazing models, and we hate them for all the shenanigans around the models, including their marketing.

I kinda wish they had not made a comeback after Claude 2.

How many benchmarks did they have to reject in the search for one that scores them top?

Is a new benchmark that useful if existing model improvements are being reflected linearly? Don't we want a benchmark that we aren't seeing much progress in.

I would love for someone to give me a coherent argument as to how this isn’t tone-deaf, vacuous garbage.

New Trust Me Bro benchmark just dropped

[flagged]

[flagged]