That is a fairly common argument, but I have yet to see any evidence to support it. In Germany, we are having a similar discussion about the Bürgergeld, i.e. unemployment benefit, which is about people abusing the Bürgergeld to the detriment of taxpayers. However, there is no actual data that show that there are a significant number of people who abuse unemployment benefits in any systematic way. The money that the state loses through tax evasion or the exploitation of tax loopholes is much higher than the money that the state loses through unjustified claims for unemployment benefits. Nevertheless, there are constant calls to further reduce unemployment benefits or make it harder to get and the argument is always something like: There is a thing that is good and benefits people but is abused by a minority, thus we should abolish the good thing.
In the UK since 2019–20, disability benefit spending has grown by 45% in real terms and incapacity benefit spending by 26%.
https://ifs.org.uk/publications/health-related-benefit-claim...
You can argue that nobody is systematically abusing the system but numbers go up. To those paying the taxes to support these benefits that sort of growth looks like abuse.