Yeah, in my opinion the point that "wouldn't it be nice if this was faster?" becomes a real issue is the point that you would feasibly need to get your day's sleep en route with today's airliners, because it's difficult to sleep for more than a few hours at a stretch on a flight even with business class conveniences (and that's before getting into the degraded quality of sleep). If I could catch a flight that's fast enough to let me hold off on sleep altogether until I get to my destination, then that's worth paying a premium even for economy seats. Unfortunately, that's also the point that a supersonic airliner becomes unworkable for airlines, because the fuel-to-passenger ratio just stops making sense. You can try to make it work with refuelling stops along the way, but that really eats up the theoretical time savings and adds its own operational overhead too.

I think we need an energy breakthrough with a denser and still cost-effective fuel before really getting into the era of supersonic transport. Maybe at some point someone will dust off the nuclear-powered aircraft designs of yore...