alt Hacker NewsI learned about the consequences of overloading the human operator when working on the primary UI for a manufacturing business. A natural inclination is to put things like confirmation dialogs around dangerous activities. I had managers telling me that one confirmation wasn't enough and that we had to add additional because people were still missing.
Eventually, we tried removing the dialogs altogether and the incident rate approached zero. If you take away the guardrails completely, it radically alters the psychology and game theory around user interaction. Imagine climbing a tall building with multiple layers of protection vs having none at all.
I strongly believe in ideas like "safety 3rd". It's not that I want the humans to be maimed by the machines. Quite the opposite. The difficulty is in understanding higher order consequences of "safety" and avoiding the immediate knee-jerk satisfaction that first order resolutions may provide.
Replies
> Eventually, we tried removing the dialogs altogether and the incident rate approached zero. If you take away the guardrails completely, it radically alters the psychology and game theory around user interaction. Imagine climbing a tall building with multiple layers of protection vs having none at all.
I think there's evidence and studies on this. IIRC removing traffic lights forces people to be much more alert, reducing accidents.
Fun fact: Bhutan is perhaps the only country in the world without traffic lights!
In litigious countries confirmations and alerts also serve as a mitigation against lawsuits.
My gripe is with multi-layer approvals for permission request tickets. If it's only 1 layer, the only layer will make sure the person it's correct. However once there are >1 layer, each layer will think the other n-1 layers will check and turn out no one will check and blindly approve things...
> If you take away the guardrails completely, it radically alters the psychology and game theory around user interaction
Cool! Did workers expect consequences for incidents? Did they get rewarded for lack of incidents?
Meaning, I imagine a world where there are no consequences for incidents and removing guardrails doesn’t lower incident rates because people aren’t incentivized to care?
Or you’re saying they naturally cared and removing guardrails allowed them to take ownership?
I’ve had to add more confirmations as people connect remotely using VNC and accidentally click buttons which then have real world actions.
To catch them all it logs the user out so buttons which affect the process are disabled, but now it can be annoying to be logged out after x minutes and always logging in.
I heard an anecdote years ago about some medical device where people were clicking through confirmations without reading, so they made the user type IRREVERSIBLE on the keyboard to proceed.