alt Hacker NewsIf the current state is anything to go by, an automated test would not only flag your out of distribution results but try to gaslight everyone reading its output with additional false indicators to map you into an area that's in distribution. Statistical models cannot accept the existence of extremely rare edge cases.
Modern LLMs routinely beat human doctors at diagnosing "extremely rare edge cases".
They have unmatched breadth of knowledge by default, and can maintain attention across entire medical histories.