The AI would confidently give him the wrong answer, since it has no way to provide the correct answer, and doesn't know its own limitations. (Or however you wish to describe "hallucinations", which is about as accurate as my description ;))

And he would think he has the right answer, perhaps write up an essay about his findings, which later AI bots will read and learn from, propgating the mistake...