There's an excellent little book in German called "KI und der Neue Faschismus" [1] (AI and the New Fascism) where the author (Rainer Mühlhoff) tries to warn about the dangers of decisions based on opaque statistical models (like LLMs) instead of a clear, human auditable decision process.
[1]: https://rainermuehlhoff.de/KI-und-der-neue-Faschismus-Reclam...
Isn't there an EU law that allows any EU citizen the right to transparency and/or appeal as to automated decisions made about them?