And if the nuggets have sufficiently small surface area then the rate of these events would be low enough to be obscured by the background from other processes. It would not be like the annihilation from antihydrogen atoms hitting hydrogen atoms. The density of nuclear matter is some 15 orders of magnitude higher than ordinary matter, so nuggets 1 angstrom in diameter could have 10^15 times the mass for a given upper bound on the interaction rate with ambient gas.
There is no way anyone would miss the radiation emitted though. It would be, at a minimum, a very perceptible ‘glow’ over the volumes of space we’re talking about.