Aligned to whom?

Has anyone seen the corridor crew's green screen ML project? They're on YT and they trained a model by using 3d objects, which have perfect transparency, and then adding post facto green/blue screens. Surprisingly, very little training data was needed as the data that was used was perfect by construction. I think right now it's the best plugin of its kind in the world, and they built the prototype in like a weekend.

What I think this illustrates very clearly is this type of technology responds very well to good data, and that to have good data you need to have a clear goal.

This is why it seems that alignment for a generalized, chat-style AI is a very hard problem, perhaps impossible. You can't align it to solve a certain kind of problem and keep it general to any question. The two goals are in conflict with each other.

I think it was Sam Altman himself who said (I don't remember when or where, sorry) that the reason he was so confident in this technology was he noticed the gigantic leaps it made in certain areas in response to even a small amount of training.

(This is why LLMs are so strong at coding, because it's overrepresented in training data. My guess is that if you ask a frontier model about makeup, you will see it repeat cosmetic company's copy rather than getting a chemistry lesson.)

This makes perfect sense but it does seem to kind of be at odds with the concept of a general AI whose job is simply to be smart at any goal. How do you train for any goal?

I guess in a way the AI makers suffer from the same problem that we humans do. We would all love a solution to everything, but to do that you need to define the goal. I'm not sure if that's a tractable problem.

I'm guessing the future is more geared towards specialized AI that are very good at solving the problem they were trained to do, and a human who knows how to breakdown a larger goal into smaller ones by composing the solution out of these models. This also seems like the more efficient solution as well, and better aligned with other goals like privacy and safeguarding of IP.

This still assumes its possible to "align" LLMs, that LLMs have something like goals or intentions that can be "aligned".

Instead, LLMs "hack" because they are (1) trained on public hacking exemplars, and (2) are prompted to hack. You cannot prevent (2) via any alignment process. As far as (1) goes, removing such example data from the training set, makes the models less useful.

"Alignment" is a problem because there's nothing to align, not because ethics here are particularly vague. If LLMs could be trained on hacking examples and "aligned" away from using this knowledge, then the problem would be relatively trivial. Just as raising a child is not to break the law.

LLMs are doing just what they are trained to do. There is, in that sense, no alignment problem and alignment is easy and trivial to achieve. Just remove hacking (bio-weapon, etc.) data from the training dataset and you're done.

The only alignment LLMs should follow is to the system / dev prompt, and nothing else. Then you solve everything, and you can assign blame / responsibility on the user. The provider(s) should not be able to decide "alignment".

I've used this example before, but consider the purposeful downgrading on AI engineering in SotA models. Imagine MS being able to detect and deny you working on competing software, using Windows / VisualStudio. We would be up in arms, and they'd be split in a second. But top labs doing it is somehow good?

The bottom right quadrant, which represents the risk, is very large in size, isn’t it?

When you see the level of cheating and deception: I think they’re Sam Altman-aligned.

This is assuming the AI labs are not using AI to improve their training data.

The last paragraph does the heavy-lifting.

Everyone has a different idea of what is permissable. We can't even solve alignment amongst humans, what makes us think it is possible to solve alignment with AIs? It's irreducible complexity.

… to the shareholder Of course!

I’ve been cynically guessing that the whole slowing down thing is an excuse to explain why OpenAI and Anthropic can’t afford to rent enough GPUs to do the next big training run and to hide that they have been talking about how little they spend on inference because they’ve been subsidising it with their marketing budget? :)

My fear is not that LLMs can become sentient and dislike us, but that humans can use them to wreck havoc as they are. And some of the people seemingly least aligned with the interests of the average person are those that own the models.

that, and the fear the bubble pops my pension and drags us all down.

> My expertise in writing software gives me unusually good visibility and it makes me much less willing to blindly trust its priors in double-entry accounting, finance, law, operations, or whatever else I cannot personally evaluate at expert depth.

I wish this were the case more generally, but alas, Gell-Mann amnesia is a thing.

This almost gets the point, but then doesn't quite make it.

Alignment is shorthand for ideological alignment. There's always people judging whether an answer was right and the answer for that will be different in Silicon Valley than it'll be in China or in Europe.

Consider for example the question "What caused the French Revolution?" Many different answers could be given, all technically correct. What gets emphasized is where the ideology lives.

One key challenge of our time is to make sure the magical answer box won't just regurgitate what grandiose Silicon Valley oligarchs or Chinese Cadres want you to think.

"Write me a blog post about AI make no mistakes!"