The book Redshift Rendezvous from 1990 explores this idea. (TBH the story is a little clunky, but if you like to see someone look at the physics in depth, it's pretty interesting.)
https://www.goodreads.com/en/book/show/1382560.Redshift_Rend...
similarly, Einstein's Dreams is a collection of very short stories that explore different flavors of time. For example: time slows as you near the city center - those who approach it end up frozen forever like the event horizon of a black hole.
I'm reading the clockwork rocket at the moment. It's a little different but it's set in a universe where the laws of physics / behaviour of light seems to be different to our own.