Two recent examples I tried to get Sol/Opus to get help with, were they gave more misdirections than beinghelpful: Tracking down the root cause of a synchronization bug in a distributed platform using CRDTs, especially dumb around sequence CRDTs. Also had less fun using Sol/Opus for building and troubleshooting fractures and constraint networks in Houdini.

I'm sure these will be much better as soon as enough data is in the training set, as always. GPT models used to be horrible at electrical stuff as well, today Astra seems pretty OK at it. It changes as they keep extending what use cases are covered by training data and what they test for and so on.