I cannot shake the feeling that with that quote the people were trying to point out a problem with something he’d said.

“With this machine, all the results will be correct, no more errors in log tables”

“And what if people put the wrong figures in?” (Hint - we’d still have the wrong results)

Babbage walks away thinking them an idiot, they walk away thinking Babbage hasn’t considered anything outside of the machine itself.