> With that said, I wouldn’t expect this wall to hold up for too long.

The models are already so good at the traditionally hard stuff: collecting that insane amount of detailed knowledge across so many different domains, languages and software stacks.