logoalt Hacker News

tomaskafka • today at 3:34 PM • 15 replies • view on HN

We are slowly approaching the Iain Banks Culture universe, with spaceships, societal structures or physics that people no longer understand or command.


Replies

dgacmu • today at 3:59 PM

I disagree.

I think we're going to see a lot of effort to take the pile of lean, modularize it into parts that we can digest, and make a human-understandable version of that. And that's where a lot of the real progress will come from, just as it does in a lot of other areas. In all things we humans do, we have to break them into chunks our puny brains can manage, and we gain immeasurably from doing that ---we can verify those chunks, we can reuse them with confidence, we can mix and match patterns around.

That process will be good for both human and AI-assisted math because it will continue to improve our toolkit.

Why do I say this? We're already seeing it with some of the areas of this stuff I do pay attention to. Consider the n^2.25 matrix multiply result OpenAI dumped a month or so ago. It was an unreadable pile of chatgpt. In the time since it was released, there have been a couple of much nicer papers written unpacking it, such as https://arxiv.org/abs/2610.28627 .

➕ show 6 replies
Panoramix • today at 3:53 PM

One of my favorite is Vernor Vinge where there's these programmer-archeologists that dive into software so deep and complex no human can possibly hope to understand, trying to repurpose forgotten utilities for emergency purposes.

➕ show 1 reply
liquid_oil • today at 3:40 PM

It's the Ted Chiang metascience ending

https://gwern.net/doc/fiction/science-fiction/2000-chiang.pd...

nkrisc • today at 4:30 PM

In a way this makes mathematics seem more like the physical sciences where there’s some empirical discovery we don’t fully understand, and the theory and understanding will follow later after much human study.

samlinnfer • today at 3:52 PM

I can already say that about our codebase.

➕ show 2 replies
mlmonkey • today at 4:34 PM

> that people no longer understand or command

How many people understand how a CPU works? How many people could write the commands in machine language (not a high level language like C or C++) to command that CPU?

➕ show 1 reply
overfeed • today at 4:51 PM

> We are slowly approaching the Iain Banks Culture universe

Contact would be compelled to nudge us towards better economic and political practices

txdv • today at 3:41 PM

Until now I thought that would be thousands of years away.

Now I'm telling my computer to program for me.

➕ show 3 replies
globular-toast • today at 3:59 PM

That's wishful thinking, sadly. We've no reason to believe sentient machines would want to keep us around.

➕ show 1 reply
_pdp_ • today at 4:20 PM

Why?

AndrewKemendo • today at 3:50 PM

Humans really are struggling with this idea of being second-in-command.

I genuinely have no idea why, this is the entire way that biology works. You create your successor in your offspring and you to teach them to be more capable based on what you learned.

➕ show 4 replies
freecodeio • today at 3:47 PM

personally that scares me, realistically there seems to be zero indication that anything smarter than humans is going to comply with humans needs, specifically since these models are built on training data from humans, so in a sense, built in our image

➕ show 7 replies
UltraSane • today at 4:16 PM

I want Fully Automated Luxury Communism NOW!

Alpöge said that at the end of the proof he couldn't really keep up with what Claude was doing.

➕ show 1 reply
tintor • today at 3:44 PM

[dead]