Mythologizing AI makes it more likely that we’ll fail to operate it well (2023)

https://archive.ph/6Nbuf

We're choosing to call LLMs (and the little "harness" programs that query them in loops and execute their output) "AI", even though it doesn't make much sense.

I absolutely love this technology but these aren't autonomous intelligences. They're little programs executing Bash scripts from JSON output.

Our ideas about AI were naive. We thought passing a basic Turing test would require human-like intelligence. It turned out to be possible with fairly basic statistical text generation, because fooling humans is easy.

It would've been nice to reserve "AI" for superior human-like intelligence capable of genuine common sense and reasoning. The irony is that the startup founders most worried about "AI" have created so much hype and funding that we may very well figure out how to build "real" AI.

Towards the end, he approaches the subject of digital provenance, and muses why it's not a part of our expectations. I find the argument compelling:

> If a chatbot appears to be manipulative, mean, weird, or deceptive, what kind of answer do we want when we ask why? Revealing the indispensable antecedent examples from which the bot learned its behavior would provide an explanation: we’d learn that it drew on a particular work of fan fiction, say, or a soap opera. We could react to that output differently, and adjust the inputs of the model to improve it. Why shouldn’t that type of explanation always be available? There may be cases in which provenance shouldn’t be revealed, so as to give priority to privacy—but provenance will usually be more beneficial to individuals and society than an exclusive commitment to privacy would be.

Remember 'View Source'? And how bundling engines eventually made it irrelevant? What if every piece of content had a genuinely accurate and useful View Source?

"A.I."[1] like "technology"[2] is a term colloquially reserved for things that don't work yet. Once something works, we have to call it something else.

1. "Every time we figure out a piece of it, it stops being called AI; it becomes just computation." - Ray Kurzweil

2. "Technology n. - Something that doesn't work yet." - Douglas Adams

>Everybody’s already using the term, and it might seem a little late in the day to be arguing about it. But we’re at the beginning of a new technological era—and the easiest way to mismanage a technology is to misunderstand it.

I've had a recurring theme where I would name a project incorrectly, and then waste weeks or months on what turned out to be an unsolvable problem. When I figured out the actual correct name for a project, the whole thing would be solved within a few days.

Naming things correctly is hard, and the consequences of failing to do that can be pretty severe. To name something correctly, you have to understand what it is.

https://archive.is/6Nbuf

AI that we have now is not a digital animal in capability or kind, and is not currently anywhere close to taking over.

But it's still in its present form very intelligent in meaningful and useful ways. And it is not too soon to talk about concerns of a potential existential threat in the future. Because it could sooner than we might realize, threaten our existence.

Because of the potential, we should have a culture of caution as we continue to rapidly improve AI.

Love this sentence

"The need to conform to digital designs has created an ambient expectation of human subservience. A positive spin on A.I. is that it might spell the end of this torture, if we use it well."

> The closest we have come to a definition of privacy is probably “the right to be left alone,” but that seems quaint in an age when we are constantly dependent on digital services. In the context of A.I., “the right to not be manipulated by computation” seems almost correct

Maybe someone can enlighten me but I really don't understand how either of these description make any sense at all. How is it not better described as "the right to decide what data can be extracted"?

What I heard him say is that people are taking away present-day jobs, hoping new ones will rise from the ashes. Yet, he also claims we need to find the creative minds who will actually create these new roles.

I'm curious: have we found those people or those new jobs yet? Is a forward deployed engineer an example of this, yet they are now doing the job of two people (sales and coding).

> “Over time, though, more people might be included, as intermediate rights organizations—unions, guilds, professional groups, and so on—start to play a role.”

Chassez le collectiviste, il revient au galop.

aka

Once a collectivist, always a collectivist.

I think the biggest pushback this article will get here is the date.

Although all he's saying is basically, "It's a tool, not a silver bullet". But the article is 3 years old and people will note that the models have been updated since then.