Not every improvement, no - if progress was steady or slowing down over time, that'd be evidence against. Instead we see what looks a lot like an accelerating growth in capability.

I think you are implying that it's invalid to consider every advance to be evidence "for", and I agree - that'd violate conservation of expected evidence. But not considering any advance to be evidence "for" is also invalid, for exactly the same reason. There has to be some news you may hear that'd make you think a singularity is more likely, and "millenium prize problem solved by an LLM" sure seems like one of those.