alt Hacker NewsStuff it's not directly trained on is going to be flaky and sucky. It was like that with programming at first too and it still is sometimes. It's hard to imagine this won't improve with better more focused training. They focus on improving "CRUD" for obvious reasons. The specialization era hasn't begun yet.
Your domain, while I'm sure it is very interesting and complex, if it proves economically interesting will be cracked as well.
Just for some context, the domain we're talking about is oil and gas and mineral exploration. E.g. At my previous job, I used to personally manage a >$400 million per year budget and that wasn't even considered significant. We had multiple >$10 billion per year projects ongoing. That was 10 years ago. The amounts are larger now.
The issue isn't a lack of economic interest.
It might be a lack of training data in addition to inherent complexity, but it's certainly not a lack of economic interest.