Yup. And we do have some degree of safeguards here-- admittedly, less in California than many other states. They are: physician required reporting of disqualifying conditions, ability for other people to report concerns about capability to drive, and the requirement to show up and undergo vision testing and not flag other concerns in the process.
There's a tradeoff between reducing the very low rate of unsafe driving by the elderly and the burden added to the very old. People over 65+ are still possibly safer, overall, than teenagers.