I love Tokyo (I spent eight months in Kawasaki in 2010), but I can see the appeal of living in a smaller place in Japan (though I’d personally be hesitant to live in a small, isolated town with little or no public transportation). Crowded trains and long lines can get tiring after a while.
I spent last summer in Toyohashi in Aichi Prefecture; I was a visiting researcher at Toyohashi University of Technology. While Toyohashi is not a small town by any means, it is far smaller than Tokyo. I found it nice for day-to-day living. Not crowded at all, and I found the bus service to be good; not world class, but comprehensive and ran frequently enough to be useful. It had plenty of grocery stores and department stores for everyday living. If I needed something unavailable in Toyohashi, Nagoya wasn’t too far away, and if I needed to be in Tokyo, I could get there in 90 minutes on a Hikari train on the Tokaido Shinkansen. I’d do it again; in fact, I’m going back to Toyohashi in a few weeks for another monthly stay.