We just write different stuff. For content driven websites you can skip js frameworks altogether. For web apps it would not be a good idea.
It's all about how much interactivity you need. A lot of business web apps are full featured commercial software packed in SPA form. You will need a lot of stuff client side and without framework it will be a mess. Unless you write framework yourself but the you are reinventing a wheel.
I'm currently working at an enterprise SPA and it also doesn't work well.
I think unless you have a very low number of features or employees (and are sure from the start to keep it this way) I don't see it working.
In my 3 previous companies I had similar issues so I think it's widespread.