There are a lot of unknowns and processes that must be undone, which often require an unknown basis. Red shift, moving bodies, gravitational lensing and other processes distort light along its journey. Sometimes we don't know the exact magnitude of a star, etc. Spectroscopy helps a lot but is limited by the fidelity of our datasets.