Maths skills have been slowly falling even before the advent of LLMs. I have a story but this is anecdotical so take it with a grain of salt.

I was in my 3rd bachelor's year studying physics (France) and overheard a conversation between two of my teachers. They were discussing how they should modify the 1st year program to now include math, because he had been noticing how more and more students were failing the more math-heavy subjects like body and newtonian mechanics. He said that they should now teach (or re-teach) calculus to 1st year students, which was not taught when I entered college (it was assumed that you learned it in high school and we would only cover linear algebra in 1st year).

I can imagine things are only getting worse with students that can now get under the illusion that they know math because they have a tool that can do it for them. Which raises the question: should programs adapt to this, like we adapted to having calculators?