Why did we knee cap CIA/NSA/DOD from nominally operating on Americans when the Soviet Union continued to push forward with it?

For what it's worth, our once-and-(hopefully)-future allies the European Union are already on board with reigning in the surveillance industry (leading by example, even). So your question is more like how can we constrain our domestic technological authoritarians when China continues to embrace theirs. And the answer is that it's not a "how". The "how" is straightforwardly enforcing longstanding concepts like personal information, antitrust bundling, unauthorized access (backdoors in sold products), etc. So your question comes down to more of a why, and the why is because it is in line with our values based around individual liberty.