One thing I would always keep in mind is that transmission of information is more expensive than localized computation on the information.

Streaming 4k video is several orders of magnitude more bandwidth intensive than UTF8 text at human rates. The fact that inference is so much more expensive than an amortized encoding of a video might actually wash out in the end.