It's a fair question to ask "who are independent executive agency heads accountable to" in a constitutional context. It is true that the Executive Branch has grown far beyond what the Founding Fathers could have imagined, but the idea of a unitary executive is that the President is responsible and accountable for everything that happens in the Executive Branch. If the voters don't like what the Executive Branch is doing, they can replace the President in the next election. What happens if voters don't like what independent executive agencies are doing? There's no democratic recourse.

Think of a scenario where a President was elected with a large-ish majority and promised during the campaign to change broadband regulations to reduce broadband prices across the country. Unfortunately, the FCC commissioners were all appointed by the previous president and block this policy change that the voters clearly support. How does that square with democratic accountability?

How can you cite “precedent” when Myers v. United States decided this issue in favor of the unitary executive back in 1926? The administrative state that exists today was only facilitated by the FDR Supreme Court overruling a bunch of precedents.

Go read the Federalist Papers. The founders thought very hard about who should exercise which powers and how they should be selected. They did not intend for 99% of the actual government operations to be run by “independent” executive officials that were insulated from elections. That’s something we made up in the 20th century in response to trendy ideas about “scientific government.”

The concept of independent agencies (that is, those overseen by Congress rather than the president) was controversial long before, and for far longer, than it wasn't.