alt Hacker NewsCivic Hygiene – avoid building technologies that could be used by a police state (2013)
Comments
The other rule here is that you cannot avoid politics if you want to create a better world. If you want the freedom to develop technology without worrying it will be used by a police state, you need to put in the work to make sure a police state doesn't happen - you cannot ignore politics and just "focus on technology."
I'm a tech worker.
I'm given a promotion for adding invasive features based on "insights".
I'm incentivized to create something patent-able to restrict competition.
I'm rewarded for more detailed metrics to reveal more.
I'm hired by how many people were affected by my previous work regardless of ethics.
There is no reward for protection, privacy, and restricting access to data.
I could be laid off at any time. I could miss payments. I could be deported.
What's a little more really hurt? Everyone is doing it.
Recently my wife got locked out of her bank account - all cards blocked, unable to log in to the website or app, so effectively her whole finances were frozen - because she was trying to order a new bank card over the phone and some system couldnt do voice recognition (which we did not know they even used). We were getting ready to get on a plane to NYC, from California (she banks with TD bank) but after a tearful session of phoning anyone and everyone we could at TD, suddenly it magically got reversed.
The problem is not just the tech, its building systems where humans are essentially slaves to the system, with no recourse or power over whats happening.
This entire thread is such a strong case for why all undergrad CS programs should have a core requirement of at least one ethics course.
As a recruiter once said to me, if you don't do the defence work, somebody else will. So you might as well do it and do a better job and get paid well. I didn't go for the job.
The automobile, the transistor, the Internet, and basically every other technology ~could~ be used by a police state. I'm not sure how that is a helpful rule at all.
I'm in the UK and it always seems weird to me how in the USA its totally normal to publically register yourself as either a Democrat or a Republican. To me seems like the kind of thing that should be private. In the UK I might choose to be a member of a political party but no-one needs to know that except that particular party. Or I can just join no party and vote in private on polling day.
Defaulting to discussing the US here: let's start with reducing the state's size - it shouldn't be the world's largest employer, it shouldn't be able to run an unfathomably large deficit, it shouldn't be able to declare war without congressional approval. Technology to run a police state is just the last step in a long line of ridiculously unacceptable steps.
The concept is like the perpetual machine in the 1800s. It is simply an impossibility. Please avoid wasting time.
Pinging the absolute jerkwads at amutable.com who state that it's not their problem how the technology will be used.
This is missing the "(2013)" at the end of the HN link, but it does seem weirdly prescient as we say a rise of more polarised politics across the World, just 13 years later.
The sad irony of "Share this post on..." with neat list of all the services doing exactly what the post warns against. They can and often are already used by police state.
I've been arguing with folks on Reddit about this lately when it comes to Google's changes to Android (where they've made it more difficult to install any software on your device without Google first tracking the identity of the developer). My argument ultimately comes down to the fact that Google (and Apple already) have built a technology that gives them immense power to control individual computing freedoms, and no matter their internal policy, they still must follow US law. No amount of promises or legit good faith on their part matter, they've made a dangerous technology and are behold to a government that is very eager to censor anything it doesn't like. It's so disappointing to see 1. Google engineers build such a thing, and 2. For people to support it.
The obvious 21st century United States instance of "don't collect data that the government might later use for persecution" is E-Verify.
Wouldn't a better title be "don't give out information that you wouldn't want used by bad actors"? Seems less about technology and more about privacy.
I'd be careful banging two rocks together else someone might start a war with them!
Has this philosophical approach ever worked? It's a commonplace in the hard sciences and engineering fields (ie, refusing to work on weapons systems), but weapons systems are elite and well-compensated engineering projects that don't have any trouble finding talent. The same thing goes for NSA, which has a line out the door from elite academia.
I'm not dismissing the sentiment here. It's morally sound. You should act in accordance with your principles and it useful to be reminded how those principles intersect the world. I have similar guardrails (probably not the same as the author's). I just wouldn't want people to overestimate what can be accomplished with this approach.
how about you guys understand that state is criminal master org. that uses violence monopoly? is it too high level or hard to understand this?
see war on drugs and more
I find it frusrating how the conversation around this almost always focuses on one side of the equation. Why is it always about preventing tools from being built in case authoritarians comes to power rather than preventing those authoritarians from coming to power in the first place? I'm far more scared off the authoritarian aspect than the tech. The Nazis didn't need Flock cameras to kill millions. It seems like focusing on the actual bad guys is going to be more effective because if the state wants to bear its weight on you, it rarely requires tech to do it.
Libertarianism has its moments but it generally hasn't caught on because state capacity is unique and useful. And virtually every last bit of it can be used to facilitate a police state because it's... a state.
Sure, we could all live without IDs and use motio belli for defence but this arrangement will be soon replaced by someone more organized because there is a huge return to that organization despite its risks.
The article is not completely wrong in its premises, but phrased incorrectly.
This is not about technologies (after all, an authoritarian government will use many existing technologies, from the car to the computer), but about powers granted to a government, that could be used differently by the next party/majority in power.
The answer to that is to not grant government too much power in the first place.
Of course, this is often a contentious point for leftists, since they do want to grant government a lot of power in the first place; they're just concerned that the exact same powers will then be used by a different political side in a way that bothers a different category of people.
Clearly you need to consider how general the capabilities and use cases of the technology is. The most general-purpose technologies will always have obvious use cases for police states and other bad groups or people.
Civic Hygiene is the opposite of how you were taught to write code...
>It should be noted that no ethically-trained software engineer would ever consent to write a DestroyBaghdad procedure. Basic professional ethics would instead require him to write a DestroyCity procedure, to which Baghdad could be given as a parameter.
"Civic hygiene isn't about saying we distrust our current government - it's about not trusting the next government." I've made similar points. Granting one government administration more power, is by extension granting it to all future administrations. Do so only with great care.
Like others have said, I don't think it's possible to create technology that cannot be weaponized. I think it's possible (mostly) to design technology so that abuse requires more people, requires the leaving of evidence / paper trail, needs checks, etc.
But still, it's not possible to prevent abuse altogether and sometimes we even applaud breaking the rules (the movie Hunt for Red October for example kicks off with the Soviet captain jumping the guardrails).
Also, the most "benign" public infrastructure is usually the most impactful, e.g., water systems and electrical systems. Don't like the people in that one town who keep protesting and voting against you? Turn off their power, water, and also make it so that their trash doesn't get picked up for a year. See how they like it.
this is an important topic, and i hate that people forget how difficult it is. in any political conversation, it's pretty easy to pull this thread to unravel anyone's adamant belief in one thing or another (e.g. gun control). my biggest concern is that people seem less willing than before to actually have those discussions.. they'll immediately just label you as being opposed to their views, "against" them.. when really we need these conversations to find viable policy solutions that wont easily be used in the exact opposite way that they were intended when implemented
Bertrand Russell:
“If war is not impossible, then every advance in scientific technique in an advance in mass murder”
As true today as then
You can replace war with “police state” and it’s the same message.
Technology is force multiplier. Everything could be used for oppression. Deal with it - the future is boot stomping on human face forever. There are lucky few that could choose whether they will be the face or foot. But it is inevitable.
Or we could stop all tech development. But it is not going to happen.
Edit; wrote this comment before I realised this is a blog post from 2013. In 2013 I would've agreed 110% with this.
-----------
This is one of those things that make for great post title and gets you all riled up at first but when you think about it, it's really shallow.
Most civic databases can be abused by any sufficiently authoritarian government. A homeless shelter's records may contain exactly the sort of information that could be used to get "undesirables" off the street. But those same records are useful for coordinating beds, outreach, medical care, etc. Surely the conclusion isn't that the shelter shouldn't keep records.
It seems much more useful to focus on limiting what gets collected, who gets access to it, and how long it can be retained. In other words, boring things like data-minimisation and retention laws. Somewhat amusingly, that gets you pretty close to the sort of principles behind GDPR, which isn't exactly universally loved around here.
I don't really disagree with the blog post's moral intuition. What bothers me is the emphasis on individual moral choices by the people building these systems. I've seen plenty of my peers happily work for companies that have spent the last twenty years building exactly this kind of technology, there will always be someone willing to build it. That's not an argument for abandoning personal ethics, but it is an argument against treating personal ethics as any form of serious societal safeguard. "good engineers should refuse to build them" means nothing.
Companies that agree to work on Chat Control deserve a special place in hell. One company that was eager to participate, hoping this harmful law would pass, is Atos. Name and shame them.
I think this runs into a basic issue that all large software systems that enforce the law contribute to locking down a society because laws tend to need be broken to be changed.
That a system can be avoided offends the investor who want their products to be ubiquitous.
The system that is not perfectly enforced offends the engineer who dislikes broken things.
Yet both are required if we as a society are to grow. I think the thing that it might be good to learn is when to say "I've done/had enough."
the problem is Capitalism does not need Democracy to function or survive
many countries like Singapore, etc. prove this easily
Authoritarianism can even be desired by Capitalists if they know who to bribe to remove regulations and get approvals
and no surprise, that's exactly where the United States is right now, teetering on the edge of that hole, about to fall in
"AI" is going to be weaponized, already is, and any resistance against its abuse is simply overriden with threats to destroy profits or even company survival at all
[dead]
Assholes like this were all for the police state back when it was up to stop-the-spread “contact tracing”, real-time “fact-checking” my private messages, arresting people for facebook posts, and transing your kids.
Now that the red pedophiles are in charge instead of the blue ones, he changes his tune.
What is the point of writing something like this?
It's embarassing for an adult to think like this. There will always be someone willing to build that thing (and that was true even before llms)
People cite Flock as an example of this. Yet the same people walk around with iPhones and Android devices that can record conversations (who talks to whom at what times and for how long), know our exact locations and log everything we do online.
If I had told you 30 years ago that we are going to build a surveillance network and make the people we are tracking pay for it all you would have laughed me out of the room. But that is what we have built. The modern Smart Phone.
Now we use our iPhones to complain about cameras on public roads as if they are watching us.
Harder than it looks. Everything is dual use. Certainly didn't expect my game development telemetry[1] system used to improve artillery [2]...
[1] https://gdcvault.com/play/1012227/Development-Telemetry-in-V...
[2] https://www.army.mil/article/85934/solution_to_weapon_develo...