This is largely poor regulation. The assumption that "more bright = more safe" and the lack of enough real-world testing.
The only other product analogy that comes to mind is "thicker = better" for hiking socks. When they got too thick, they applied too much pressure to the heal and also provided additional moment distance making it far easier to roll an ankle.
The larger contrast of very bright lights can actually make visibility worse. In urban planning it's well known that having more smaller lights is always best for safety and visibility than having a few very powerful ones. I'm not sure how that can translate to car headlights but certainly we're all less safe when car headlights are blinding everyone and SUVs with comically tall bonnets are becoming the norm in the US
I believe that it's more nuanced than that (this is from the context of German regulations):
In the old days, reflectors and diffractors were rather crude and light distribution wasn't even at all. Regulation was setting a light intensity limit for the arc above the cutoff, assuming that the brightest spots would be just that, unintended brightness outliers. And when those would comply with the limit, the typical brightness hitting the eyes of oncoming traffic would be much lower.
Now reflector design is SO much better manufacturers striving to make their buyers happy can make lights pushing out photons exactly at that old regulatory limit over the entire cutoff area, and with a precise jump where the cuttoff stops. That's awesome for the person behind the wheel ("best lights I ever had!"), technically within the old limits and a terribly blinding for everybody else.
Car headlights are regulated. My hunch is that the regulations are based on technical background that is not up to date with modern light sources including LEDs and HIDs.
Styling works against us too. The ability to control the geometry of the light beam improves with the size of the optics relative to the emitter, but people want a car with sexy little lights.
I designed optics for lighting in a past life, though never for an automotive application. This issue is actually on my radar because of the blinding brightness of bike lights on the local bike paths.
I think the problem is even worse because the mentality quickly extends from "more bright = more safe" to "drive faster."
Do you have any sources you can share about sock thickness and ankle injuries?
Yeah but more bright is more safe ... for me driving around with the light of a thousand suns coming out of the front of my care at night it seems pretty safe, "good luck everybody else".
Yes, and this poor attitude of "safety" meaning "safe for the driver" extends to all sorts of terrible safety regulations.
41% of vehicle deaths are people not even in a car[1]. Yet car safety regulation is heavily focused on the 59% that are, nothing to regulate the ridiculous gender-affirming hood heights or aftermarket lifts that turn a survivable collision into a deadly collision.
[1] https://crashstats.nhtsa.dot.gov/Api/Public/ViewPublication/... Table 1, paragraph above