It seems expectedly error prone.

Aside from the general limitations of this technology, Google needs this to be quite cheap if it runs for every request.

There is not a lot of revenue for a single search, and right now the AI results are actually pushing the links people are paying Google to display further down the page.