IMO Bun and Vite are best suited for slightly different things. Not to say that there isn't a lot of overlap, but if you don't need many of the features Bun provides, it can be a bit overkill.
Personally, I write a lot of Vue, so using a "first party" environment has a lot of advantages for me. Perhaps if you are a React developer, the swap might be even more straightforward.
I also think it's important to take into consideration the other two packages mentioned in this post (oxlint & oxfmt) because they are first class citizens in Vite (and soon to be Vite+). Bun might be a _technically_ faster dev server, but if your other tools are still slow, that might be a moot point.
Also, Typescript also "just works" in Vite as well. I have a project on work that is using `.ts` files without even an `tsconfig` file in the project.
Worth mentioning that both oxfmt/oxc are in alpha. I would put money on them replacing prettier and eslint, but they're not ready for production yet.