I feel 100% identified with you. I am working on a non ficton book about a niche topic and I wouldn't do it for the money at all. It's about the "prestige (or perceived prestige)". I am about to finish the first 1/3 of the book (the first draft, anyway), and I am already attempting to reach out to publishers to see if they would be interested in the book (at least the ones that don't require a literary agent!).
Some of them already replied saying the proposal seems interesting but they want to read a few chapters. I don't know if I am in the right path or not, but I'd love to read more about your experience and what can be shared!
That's awesome, can I ask what the topic is? What I did for "selling" the book was to create a proposal -- about 45 pages that has a skeleton outline of each chapter (it changed significantly during the end writing process but gave the publisher a feel for the topic), a sample chapter, and some more sales/marketing details like what are comparable books (and how well they sold if you have that data), who your audience is and how you plan to reach them (OP was in a great place having a following), why you're the right person to write the book, etc.
I'll drop in from the sidelines, with the massive track record of having written one (1) book.
The prestige probably isn't what you'd expect. Having an ISBN to one's name carries ~zero weight for the people that actually matter for your career (it may mildly impress some future coworkers in a decade's time, though). The real value of having written a book is that then you have written a book.
Having a publisher carries one extra benefit that was merely implied in the post: you get assigned a professional editor. If you're lucky (I was), the editor has a really good understanding of how to wield language and the lessons you get from the editing process are going to far outweigh any direct financial benefits. When I wrote mine, I had been doing freelance writing for a large IT magazine for nearly ten years - and as a direct result had been taught how to use written language as a weapon by a good number of old-guard journalists. The year I spent on the book project taught me a LOT more still, because I was assigned an editor who herself was studying (in a university) to become a language teacher.
The skills I picked up from that process are with me to this day, and ironically have been the single most valuable asset I have as an engineer and/or engineering leader. Being able to write well to a varying audience is a superpower. You also learn to appreciate professional authors, because what they do is decidedly not easy.
In the end my book sold well enough to earn out its advance, so I guess it was a non-failure for the publisher as well. I also picked up a lesson for all aspiring authors:
Writing a book is easy. You sit down by the keyboard, slit your wrists and pour it all out.