AI inference is basically 1 operation iterated over every single model weight. That's what's makes computer in memory attractive and feasible.

General purpose workloads would require much more silicon real estate and are not (typically) nearly as bandwidth hungry. Just like not all CPU workloads benefit from being adapted to a GPU, even fewer will benefit from being adapted to processing in RAM.

I looked at this for a client close to 20 years ago. The problem is that you need to get software support for it. Arguably easier these days with open source but still doesn't magically happen.