The paper you linked claims on page 10 that machines have been performing comparably on the task since 2012, so I'm not sure exactly what the paper is supposed to show in this context.

Am I to conclude that we've had a comparably intelligent machine since 2012?

Given the similar performance between GPT4 and O1 on this task, I wonder if GPT3.5 is significantly better than a human, too.

Sorry if my thoughts are a bit scattered, but it feels like that benchmark shows how good statistical methods are in general, not that LLMs are better reasoners.

You've probably read and understood more than me, so I'm happy for you to clarify.