This certainly has been in the making for longer than the "everything we do must be for AI" bubble. In fact s390 has its own on-die inference engines and they have access to the same caching mechanisms as the main processor (which are quite insane).