Even with altitude, you still need time. The Earth moves around the Sun, the Sun around the galactic center, all at hundreds of km/s. Without a timestamp, lat/long/alt just tells you where something was, not where it is. Time was never optional.

You can model geography as a 2D heightmap to a pretty good approximation tbh.

Hmm I thought that, but we don't really live in a 3D world (or use the altitude parameter in a very meaningful way in life) so I wondered whether there's something else I was missing.