Thank you for your feedback. Yep I definitely understand that Video.js is just the front end. I want to avoid using Wowza / Red5 and just want to serve chunks of video files, essentially, buffering them and pasting them to the "end of the stream" laying down tracks ahead of the video.js train riding over those tracks.
So I'm just wondering whether we can do streaming that way, and video.js can "just work" to play the video as we fetch chunks ahead of it ("buffering" without streaming servers, just basic HTTP range requests or similar).
You should check out HLS and DASH. If you're already familiar and you're not using them because they don't meet your requirements, then apologies for the foolish recommendation. If not, this could solve your problem.