If you work at the factory you should be able to complain about the boss when he's out of earshot without him snooping.

If that's something he cant handle he might have a problem with personal accountability.