Well, if we assume that a point particle is an infinitely small thing with all of its charge concentrated in one point, the dirac delta function is obviously the correct way to describe that. Of course, there is not really a way to find out whether that is true. Still, the delta function makes sense if something is so small that we do not know its size. This idealization has, however, led to problems in classical electrodynamics: https://en.wikipedia.org/wiki/Abraham%E2%80%93Lorentz_force. Search for 'preacceleration' in this page. This particular problem was ultimately solved by realizing that in this context quantum electrodynamics is the theory to applies. But, then again, using point particles also causes problems, namely the need to renormalize the theory which is or is not a problem depending on your point of view.