alt Hacker NewsFTDI (the company practically synonymous with TTL serial adapters) uses 3.5mm tip-ring-sleeve connectors for this. In other words, a common headphone jack.
I added one to my single-board computer enclosure, following FTDI's wiring. Now I can easily connect whenever I need to use the serial console, and a standard 3.5mm audio extension cable will let me reach across the room without moving my main computer. Replacement parts, if I ever need them, are cheap and easy to find.
Here's the pinout:
https://www.ftdichip.com/Support/Documents/DataSheets/Cables...
That really sucks for hotplugging since TRS connectors sliding in basically make random connections before seating properly.
Granted, you shouldn't hotplug TTL serial, but everyone™ does it anyway. (In some situations you're even forced to, to avoid reverse powering something.)