We only recently started requiring ID here in the UK, it was a move that was seen by many as a form of Gerrymandering because the actual recorded rate of voter fraud in a general election is very low, around 200-300 cases per election usually.
In the last general election 16,000 people were prevented from voting because they didn't have the correct ID - despite there being mechanisms for them to get suitable ID for free ahead of an election.
Sometimes the fix is worse than the actual problem you're trying to solve.
Interestingly UK has another system to prevent fraud - you can only vote at a single polling station. So it is historically more restrictive than most of US (and here in AU). Explains the low fraud rates. And likely inhibits a lot more voters than a simple ID check.
Also, without a photo ID the fraud rate is near impossible to measure - people can just sell their votes to imposters, fabricate voters, etc. It would be undetectable in your stats. A similar concern applies to postal votes, and there have been high profile cases of this in the UK.
As a concrete example The Tower Hamlets Scandal saw Luftur Raham's election declared null 'on the grounds of corrupt and illegal practices by him and his agents, and general corruption so extensively prevailing so to reasonably supposed to have affected the election' despite a similar number of low hundreds of allegations of fraud. That's a proven-in-court example showing you are wrong to dismiss the concern based on the data. The methods used are clearly not detectable by the system you endorse, and that system has demonstrably lead to the unjust disenfranchisement of your fellow countrymen.