Ask HN: AI Panic, you buying it?

I don't consider this a "yes/no" question. It seems quite clear to me that the idea of "AI doom" is neither certain (100% probable) nor impossible (0% probable). The question is where on the continuum is the truth. And I don't know. And I'm pretty sure nobody else does either.

That said, there are plausible scenarios where AI can cause great harm, even if it doesn't extend to "eliminating all humanity". So I don't reject calls for putting more emphasis on AI safety and alignment out of hand. On the other hand, I don't support the idea of implementing a new "Turing Registry"[1] like government agency. I also don't think most of the various proposals out there for regulating AI are enforceable in the slightest.

So where does that leave me? Uncertain. I wish I could take a more concrete stance, but from my perspective, the data needed to do that just isn't there.

[1]: https://en.wikipedia.org/wiki/Neuromancer

No. Frontier AI companies need the government to shield them from becoming commodity. That's why they're begging for regulation.

On the other side, the current administration also has powerful backers whose businesses are complemented by AI and would benefit from it becoming a commodity. Think about who Vance is speaking on behalf of.

So the AI companies need to use fear to drive public demand for their regulation efforts. IMO there's little to no honest discourse and a lot of attempts to manipulate public opinion for objectives that we have little stake in. I personally don't trust any of the involved parties to make decisions for the public good.

No. At least not with an LLM. Maybe in 50 years with real AI. A lot of non technical people entering the conversation which is very annoying.

I'm not buying it and nobody should. It's just a marketing ploy, and most importantly a way to keep the AI bubble afloat.

The fact is that AI is incapable of living up to the hype and AI companies can't keep promising that any day now it's going to cure cancer and become self aware and be as smart as humans and let employers fire most of their staff. They know that isn't going to happen. They want to be able to blame the Slowdown for AIs continued failure to deliver while also claiming the Slowdown is needed because of how successful AI is.

That said, AI isn't harmless. It could be harmful to the internet since the flood of bots and slop could drown out the voices of real people. AI will be used and misused by companies and governments to screw over people and hide discrimination. People's air and water will become increasingly polluted while their already difficult to afford energy prices will go up. AI will enable new levels of surveillance and control at a scale that was never possible before and with a lot of false positives and negatives.

We don't need an "AI slowdown" so much as we need to rethink what we're really gaining in exchange for what it's costing us and what steps need to be taken to protect the public and make sure they get prioritized over private profits.

No, this all reads as some PR stunt or marketing ploy to me. All news is good kind of thing.

No, just a push for regulatory capture, nothing more

I'm not panicking but there are very real risks.

If you haven't looked at the stuff that has come out in the past couple of months (like the Huggingface hack and the Navier-Stokes solution) and/or you're dismissing it as marketing, you're grossly misunderstanding the current state of what's possible, what the potential of the frontier models deployed at scale is. Personally I'm finding it fascinating to follow, and I'm much more excited about the upsides than thinking about the risks but the risks are very real.

No, I don't buy into that at all.

I think that the genAI companies themselves present a greater danger to us (arguably not an existential danger, though) than genAI itself.

All of the following are true:

- They have internal models that still show there's gas in the tank, in terms of improvements

- Open models are eroding their customer base on the low end of the curve, & also you need not use Astra Max to classify images.

- The true demand for high-intelligence tasks is lower than what they need to justify their spending

- It would be wonderful for them to kick up a scare & give reasons as to why high-int open models should be regulated out of existence, opening up the lower end for them to take again

- Setting up an oligopoly for themselves gives them a captive audience, just like it did for (insurance, banking, telecoms, etc.) via regulatory capture

Personally no, I think its a way to protect the market from crashing. It costs way more than it generates and they cant validate it to their shareholders

I would benefit from lower ram prices so I would love to see the AI panic result in restriction on the datacenter build out.

If you look around I don't think we're at the point where even if AI go rogue it could just disrupt us for a while. No real major concern to humanity but yeah I guess having rules and regulations would be plus point. So for now there shouldn't be anything to panic about.

They’re not giving any details about why or how.

If they were the moral people they say they are, and believed what they say about it killing humanity they would stop unilaterally. These are the “rationalists” but they act like the fundamentalist church I grew up in. Every fact serves the same conclusion.

This reminds me a lot of Y2K. There were dire warnings about planes falling out of the sky, financial system meltdown, all systems being broken, etc. In the end nothing happened.

You could argue that's because of the prep work people put into avoiding that. The same thing will happen here. The HF incident was a warning sign and people will put better guardrails in place as a result.

Astra, Claude -- these aren't "Jesus on a keyboard" somewhere. They don't assemble into one big brain and then answer everyone at the speed of light.

It's weights. Spun up billions of times over.

If humans are taking the outputs as scripture? As "intelligence we must obey"? Yeah, we're in for a bad time. "Intelligence we want running core infrastructure?" Puke.

But why would we do that? Why would we hook up an LLM to a robot factory, or make an ASI that could easily move itself into the ether?

Everything magic about LLMs is what harness we put them in and let them drive. There's nothing dangerous about predicting the next token. There's something dangerous about letting them do dangerous things, but nobody seems to care for maybe Tesla FSD. We don't think it's about to take over.

Any assignment of motive or whatever to the AI, you're talking about one cobbled-together harness or something, with an LLM manning the controls. You can swap the brain out from Claude to Astra and the motivated-AI will maintain its motivation--it lives outside of the inference or the data center. So yeah, let's not let it take control of the nukes. That would be really dangerous. But it's not going to take control on its own. "It" is a calculator.

Anthropic obviously has guardrails to prevent people from using AI to do all kinds of things already. All I hear about is you guys bitching it won't do your barely-related-to-security work. That's all we need to continue doing to have AI safety. Stop humans from doing evil and stupid things with it. The idea that the datacenter is going to assemble its own sentience and motivations is pure retardium (at this time, for llms).

idk about the marketing ploy, but the latest mainstream media coverage is giving covid panic vibes. which is making me think it'll be fine. it's also entertaining how the rationalists who work for the AI companies are being extremely irrational with their fear

I think there are undeniable and novel risks with AI. But there are risks in everything. We could theorize all day about them. We all use tools every day that could be used for very ugly acts. An iPhone can be used to plan a terrorist attack. A 3D printer can be used to print weapons. We could sit here and fear monger all day. Time will be needed to understand, weigh and contextualize these risks.

What is a more immediate concern for me is the efforts of frontier labs to have government craft legislation that shields them from competition, cements them as monopolies, or assigns them the capability to be moral arbiters in our society.

From my perspective it looks like working to deep in AI makes you crazy. No other way to explain all the weird bullshit these billionaires throw around.

Sure AI sucks, but not to that degree (yet)

Nope. They know their business model is not sustainable, so they are trying to prop it up before their IPOs.

if AGI will come in versions i think we are safe

at this moment? no. one day, perhaps soon? yes

[dead]