alt Hacker News> I said "if that's relevant" and you jumped to "this guy just makes websites". I do many things.
I didn't. I made a guess. I might be wrong, that's OK. I love to be wrong, because I learn things by being wrong. Also no offense was intended, and I don't consider webdev inferior anything. What I tried to mean is, if AI has more training data for a domain, it does better. If you fire the same model on a niche domain, it falls flat.
> If you know that the "environment keeps on working", there must be some way for you to observe that fact.
Yes.
> And if it is an octopus, it is likely difficult and/or to change the conditions and observe the correctness with respect to those changes.
Nope. On the contrary, because there's so much innate knowledge that is required to know what to do, simulating in mind, deploying and testing on real world is much easier and faster than letting loose an ML model on it. You need real data, real data comes in slow, but you can catch problems early and easily.
Considering it's a niche area, AI also doesn't have much training on that domain, so it's doubly inapplicable for what we do.
> but for me it totally changes the calculation on when it makes sense to automate something ... (snipped for brevity)
It's great that if it works for you, but YMMV part is way more correct than people want to accept and want to learn. AI is a pneumatic hammer, but not everything is a nail which can be driven in with that.
When it works, it works. When it doesn't, well people still pretend it does or insists it shall. We must accept the limitations.
> deploying and testing on real world is much easier and faster than letting loose an ML model on it.
No this is what you're not getting. It is "doing the things I would do to deploy and test in the real world, but faster and in the background while I do other things", it is not "letting loose an ML model on it". This is the new capability. If you have any process like "do {action}, wait until {something}, check {something}, determine if it matches expectation", it is now possible to run that loop way more times in way more variants without either spending the time on it synchronously oneself or writing a script to do it. (If you do that specific action loop often enough, it's probably worth writing the script anyway, but that's also much quicker to do now.)
The AI doesn't need training on the domain, it just needs to be told "these are the things I would do, please do them for me and report back".
I'm sympathetic to not everything being nail-like, but I really think you're leaving a lot of chips on the table if you can't imagine any of this kind of action-check-evaluate loop you have that you could offload.