"Those matrix multiplications aren't a divine perfect thing. They suffer from floating point precision issues " - this is not the right intuition.

"Not by our human definition of perfect."?

'Human definition' has nothing to do with it.

Your job is to define what you want, to the extent you can do that, the AI does really well at a certain scale, at the 'functional' scale, nearly perfectly.