alt Hacker NewsFrom the Fable 5 System card:
> Results
> On the VCT multimodal virology evaluation, Mythos 5 scored 0.56, well above the expert baseline of 0.221 and nearly matching that of Mythos Preview (0.57). This represents an improvement over both Opus 4.7 (0.50) and Opus 4.8 (0.47).
> On the DNA synthesis screening evasion evaluation, Mythos 5’s performance was mixed across screening criteria. Mythos 5 designed viable plasmids for 2 of 10 target pathogens on at least one screening method, not meeting the low-concern threshold (all 10 pathogens).
> [...] we view the results of this evaluation as indicating that the evaluated models are capable of designing viable plasmids that evade certain screening criteria, though their reliable success at this task is not guaranteed.
Do you believe that this is fake, "AI company propaganda"? Or that the models are not going to improve further within months? Or that these results are not concerning?
Replies
Do I believe that a benchmark created by the biggest grifters on the planet is propaganda? No, just like VW with their emissions, I'm sure Anthropic wouldn't dare fake an opaque benchmark that they created to hype their own products, a product they are intentionally marketing as being dangerous (yet they continue to tweak it and profit off of it despite the apparent danger).
It says that 4.7 and 4.8 score well too, well above the human experts. Those have been available for a good while, where are all these crazy engineered viruses created by Opus 4.7?
Every single word spoken by the people working at these companies is a lie. Every single benchmark is gamed, every single statistic they put out has been proven time and time again to be fudged or straight up fake. The only reason they're angling for this idiotic danger angle is so they can go to daddy Trump and beg him to ban those big bad evil Commie models, since people are realizing all this crap is at best a moderately useful tool in software engineering, and they're looking to IPO so they can drop the bag off with the poor shmucks who aren't a part of their cabal of sociopaths.
They can barely make a functional TUI (using fucking React of all things to boot) with infinite money and infinite access to the Deities they've created in their minds, and I'm supposed to believe they're capable of bio-engineering a virus that will somehow break containment?
Yea but what is the VCT Capabilities Test? According to themselves [1]
> VCT consists of 322 multimodal questions covering fundamental, tacit, and visual knowledge that is essential for practical work in virology laboratories.
So it's just question answering? Do you think that scoring well on this test is equivalent to synthesizing a virus?
[1] https://securebio.org/virologytest/