I'm thinking of a time I worked a university help desk and had to deal with a professor that considered himself a night owl and didn't understand why we weren't staffed during his preferred working hours of 10pm-8am. He was literally the only one on campus working those hours, besides things like security.
There's wanting to work a bit earlier or a bit later than everyone around you but there's a limit to how out-of-sync you can get.
For work it doesn't really matter. If your company is large enough you'll have offices around the world and have to deal with some colleagues not being available during your working hours. Just keep all communication asynchronous.