Not only builders, the greatest takeaway for me is that everyone has a responsibility in shaping the discourse, culture, and usage of transformative technology. This "the builders will do the right thing" mentality is even (in my interpretation) explicitly called out in several places:

> It is the pursuit of the common good that gives life to a people, understood not as a mere collection of individuals, but as a living reality in which people learn to recognize that they themselves are interconnected and jointly responsible for the res publica. In this sense, every person contributes to the building up of one’s people...

> When it comes to decisions regarding economic flows and digital platforms, as well as the governance of data and algorithms, we cannot allow a handful of actors to dictate these processes on their own; instead, we must build forms of cooperation that respect the various levels of the global community and make them jointly responsible for the common good.

> We cannot be satisfied with merely calling for the moralization of machines — the so-called “alignment” of AI with human values — without also having the courage to insist on a further condition: the possibility of openly discussing the ethical frameworks involved and subjecting them to shared standards of social justice.... What is needed is a more active political involvement that is capable of slowing things down when everything is accelerating, and of protecting the opportunities for communities still to be able to participate and ask questions.

Not new for software and hardware industry though, practitioners have just chosen to ignore it. From the Association for Computing Machinery, which encompasses all forms of software development, the very first principle is the public good:

"Software engineers shall act consistently with the public interest. In particular, software engineers shall, as appropriate:

1.01. Accept full responsibility for their own work.

1.02. Moderate the interests of the software engineer, the employer, the client and the users with the public good.

1.03. Approve software only if they have a well-founded belief that it is safe, meets specifications, passes appropriate tests, and does not diminish quality of life, diminish privacy or harm the environment. The ultimate effect of the work should be to the public good. ..."

From the IEEE, which also encompasses computer engineering, their first principle and its first few sub-items are:

"To uphold the highest standards of integrity, responsible behavior, and ethical conduct in professional activities.

1. to hold paramount the safety, health, and welfare of the public, to strive to comply with ethical design and sustainable development practices, to protect the privacy of others, and to disclose promptly factors that might endanger the public or the environment;

2. to improve the understanding by individuals and society of the capabilities and societal implications of conventional and emerging technologies, including intelligent systems; ..."

Cool, inspires me to think, "should we regulate away the toxic side effects which we realize COULD have come, if we built it less considerately?"

Yes but... No one living today knows what direction AI technology will take humanity. If we have an algorithm breakthrough then we may avoid building new data centers. If the abilities of the technology plateau then there might not be large impacts on employment. Builders need to focus on the impact of their next steps. Don't put polluting natural gas generators in neighborhoods today. Don't make unemploying folks the goal of your tool, today. Don't make decisions that harm people and the environment today.

> Therefore builders "bear a particular ethical and spiritual responsibility"

> This is a message we need right now.

Feels good man. The solution found by the private parties driving technological change is sainthood. Or aiming for it. At least, better than you. They have the vision of what's good for the herd, but the more time I spend as a sheep, the more it seems the "herd" is just a way to recycle the story of their own exceptionalism stripped of any mark of individuality. A simple visit to fiftyyears.com will greet you with "We back the indispensable". I guess it's the same "we".

hard disagree. No matter what gets built, it will be used for bad things if the society is not properly constructed. We need to fix our government and then you can build anything you want. I don't like putting the blame on the builders, building is already difficult as it is. Also you can't built something in isolation, word will get out anyway. That's why this speech is either naive or he just wanted to tick a todo list: say something about AI.

> Scientists are actually preoccupied with accomplishment. So they are focused on whether they can do something. They never stop to ask if they should do something.

From Crichton's book Jurassic Park, which like most of his books is about the perils of technological advancements.

They used the quote in the movie, slightly tweaked.

I have only read a few passages (and some of the excellent quotes others have shared here), but I find the underlying message here so much more compelling than those found in the various "manifestos" which come out of Silicon Valley.

I think reading this helps me imagine a version of the future I'd actually like to live in. A version where technology is used well (rather than preaching for abstinence from technology) and where values other than "intelligence" (in whatever guise) are on an equal footing.

Even writing that makes me feel naive (and to an extent I know it is) but I think it would be inconsistent for someone who cheers for humanity's efforts to solve/chip away at "impossible" problems (like LLMs were thought to be not so long ago) to shirk from the challenge of making the world better for _everyone_.

Not from a church which indocrinates young people and has a massive child fucking scandal in the running for the last 15 years.

Science and Engineering is morally detached.

Ask a scientist or engineer what philosophy or theology has taught us about the source of morality and their education, training and experiences havent prepared them to answer that question.

This didnt matter so much in the past because their activities never had the scale it does today. For basic training in philosophy if you are mid or upper level exec whose decisions are going to effect a whole lot of people, go to open yale courses and take the intro to Philosophy classes. It will help develop your answer to the question - why do you do what you do if you are going to die anyway tomorrow.

I would also recommend God and Golem, Inc., A Comment on Certain Points Where Cybernetics Impinges on Religion:

https://monoskop.org/images/1/1f/Wiener_Norbert_God_and_Gole...

Norbert Weiner was an atheist but he talks about three areas religion is the only thing to have really examined that relate to capable AI: omniscience, omnipotence, and worship (gadget worship). It has very prescient stuff on blackbox learning/distillation, reinforcement learning/reward hacking, alignment through human feedback.

His The Human Use of Human beings and Cybernetics are extremely good too and have more of a mash of the themes between Rerum Novarum and Magnifca Humanitas, and more near-term automation.

[flagged]