Sample it with what? An infinite precision ADC?

This is how old temperature-noise based TRNGs can be attacked (modern ones use a different technique, usually a ring-oscillater with whitening... although i have heard noise-based is coming back but i've been out of the loop for a while)