Point clouds usually are not equidistant or arranged on a regular grid. You are right that filtering Lidar data to only get the terrain does create gaps in the cloud. The conversion of the point cloud to a surface does deal with holes though (that's the whole goal), and it doesn't matter if some are larger than others. There are different methods to do so, for example delaunay tetrahedrization [1], or ball-pivoting [2]. You do get flat areas though, but that's often a good compromise to make.

[1]: https://en.wikipedia.org/wiki/Delaunay_triangulation

[2]: https://en.wikipedia.org/wiki/Marching_triangles