Usually the answer is that it creates more problems for more people than it solves. The work involved to add options to allow full access is substantial and could increase the attack surface.
Features that are expensive but only useful to a small portion of the userbase often don't get prioritized.
Features that are useful to competitors who would like to provide an alternative to a Google product also don't get prioritised, it seems.