As someone who thought the same (stock was enough for me) until I had some free time and switched, here are some things I'm really enjoying, in no particular order:

- Format support: EPUB, CBR/CBZ (comics, graphic novels), PDF

- Dark mode

- Fully programmable gestures and shortcuts

- Automatically syncs with my readeck instance (think readitlater/instapaper) to dowload the articles I save for example here on HN

- Calibre integration over wifi

- RSS/Atom

- Custom dictionaries

If you are into manga, there is an extension for directly downloading them on the device

I'm really happy I made the switch.

Being able to put epubs bought elsewhere and reading them without needing to convert the files is a huge plus

Stuff which may be above the kindle stock features (as seen on Paperwhite 10):

- Allowing to highlight text across pages. Used to work in the past but no longer works

- Allowing exit to library from mobi files without a cold reboot of the device. Used to work, no longer works.

- Not deleting contents of my library on an update overnight? (files still on disk, metadata gone, no longer visible on Kindle)

- consistently working dictionary, offline, at least for English texts? (used to mostly work, with recent updates it is broken)

- Not getting worse on every update

- Better keyboard layout, maybe supporting smaller languages

- Not deleting purchased books remotely

- better bookmarks and navigation support. I find Kindle useless for the technical reading and learning materials, as navigation and cross-referencing is slow clunky and useless, I need paper books for that, or a computer.

- Better image display? Also hinders proper use for digesting technical content.

- Text Search?

- EPUB support.

- Better and faster UI. I always thought the Kindle was just slow, nope, it's just really bad software

- Customise any/everything. Font size, line spacing, paragraph spacing, margins, header/footer content, the list goes on

- Ability to use things like BookOrbit (or alternatives) to manage your books and sync your progress/stats.

- LLM-integration through plugins like KOAssistant, highlight a word/phrase/passage to get it explained to you. This was invaluable for me when reading through a series from the 1950's about the late 1700's, lots of phrases I'd never heard. Many I could grasp the meaning from context but it was way more fun to hear the "why" behind the phrase. Same with historical events that I am less knowledgeable about than I wish. Let's me stay "in the book" instead of picking up my phone.

- Full control, some people don't care/need this but I love being able to SSH into my Kindle and do whatever I want. I love having my entire book reading experience end-to-end be fully open source and self-hosted.

IMO the most important one is more options for PDF rendering, often allowing you to read PDFs without squinting.

Well tbh the reason I like it has to do more with calibre (free software)

Never updating if you don't want an update. I see no reason for my e-reader to have updates. I have a Kobo and after I updated it once about 5 years ago I haven't since because it already does everything I need it to.