alt Hacker NewsRegardless of the computational needs to facilitate mass surveilance, there are inescapable storage needs precipitated by surveilance. Governments want to retain data because its value in the future is unknown in the present. This is the entire premise behind the Harvest Now Decrypt Later[0] principle that fuels much of the previous surveilance actions taken by governments. But HNDL is speculative; it either hedges on the bet that computational breakthroughs will weaken encryption methods to the point where brute force attempts at decryption are not financially feasible for most actors but still physically possible, or rely on a backdoor in an encryption algorithm to decrypt data that is valuable. There has been much speculation about this feasibility and value RE: quantum computation and also the design of encryption algorithms from the ground up (see things like the Clipper Chip and RSA's relationship with the NSA).
With things like video, photo, audio, and liturgical surveilance, the data is largely out in the open (or simply purchased from their hosting platforms, leading to social media and adtech companies like Facebook and Google being defense contractors by nature) and does not need to be decrypted (in most cases) - rather, it needs to be correlated. This is a great task for machine learning because that's what the field has largely been engineered to do for the last 60 years, but it relies so, so, so heavily on having data to both train on and use in its practice. Storage of all data is the perfect end-goal of a surveilance state. Storage requires data centers. Data centers of this calibre are effectively black sites in that they are nigh impenetrable in both practical and legal senses, and I'd wager many data centers host actual digital black sites simply due to their ubiquity as means of doing so coupled with the necessity of the hardware and facilities they provide that are needed for doing so.
Theres really nothing that will ever be publicly available that would say "this data center has a hundred thousand SOTA GPUs in it that are doing gait recognition on every camera in every airport in the world in real time" versus "this data center is a bunch of tape machines that store everything ever recorded from every facsimilie of Room 641A[1]".
0: https://en.wikipedia.org/wiki/Harvest_now%2C_decrypt_later
Harvest Now Decrypt Later principle holds very well for intelligence work, the harvested information could be useful for very long time.
You can see this in the long time intervals specified for Declassification of classified documents.
"Executive Order 13526 establishes the mechanisms for most declassifications, within the laws passed by Congress.[1] The originating agency assigns a declassification date, by default 25 years. After 25 years, declassification review is automatic with nine narrow exceptions that allow information to remain as classified. At 50 years, there are two exceptions, and classifications beyond 75 years require special permission."
https://en.wikipedia.org/wiki/Declassification#United_States
"Decrypt Later" should generalized to "Use Later", many recordings are useful without decryption, just from the metadata you can establish who, when communicated. Identification of relations between persons, building of social graphs, is the basic of intelligence work. Much of this relationship analysis can be done just from metadata.
One example of such analysis is the NSA Co-Travel Analytics
"CO-TRAVELER does not simply collect location information. It creates a portrait of travel times and people who crossed paths, revealing our physical interactions and relationships. The cell site information goes beyond email and phone calls and ordinary telephony data, allowing the U.S. government to know who we are with in-person and where. This is information that would be impossible to collect using traditional law enforcement methods. "
https://www.eff.org/deeplinks/2013/12/meet-co-traveler-nsas-...
https://www.eff.org/files/2013/12/11/20131210-wapo-cotravele...