Honestly at first I was really impressed by AI progress because most of what I do day to day falls squarely into the training distribution - and the models seem to excel at that kind of work, literally in certain areas they are more competent than me.

But then I started using the same models for some custom work and in situations that are trivially relatable you could see 0 skill transfer unless explicitly hand held through the process.

For example I built a custom GUI toolkit for my 3D engine, GUI is defined with React components so it should be super familiar to the engine, the layout primitives are very standard and support the stuff needed to make the UI I expected - but the results I would get out of both Opus 5.5 and Astra are garbage. No padding, shit alingment, etc. stuff that it oneshots with web components/frameworks. I prompt the model to "make it look better" without specifics and it's utterly lost - you can see there's no understanding of "design" there's only "trained on a million of HTML/CSS web layouts to spit out what the most common layout patterns in the training set are". When I give it the specifics like "go to x css framework, take the default layout guidelines like sizings/paddings/colors/etc." then it gets closer to what you'd expect but if it truly undrestood design it would realize that text snapped to a vertical divider line is shit design that needs to be fixed when I prompt it to make it less shit.