I’m much more optimistic on device-side matmul. There’s just so much of it in aggregate and the marginal cost is so low especially since you need to drive fancy graphics to the screen anyway.

Somebody will figure out how to use it—complementing Cloud-side matmul, of course—and Apple will be one of the biggest suppliers.