I think without a doubt that will be the case, unless the trend of compute getting better over the last 60 years suddenly stops. It should become less tough to run a local model and our devices should become more powerful.

However I think there is still a significant runway for these models to scale, so there will always be some sort of offering from providers. I can't imagine that our current use of the context window will be how that looks in a handful of years.