Though it was years ago now, I did spend a couple of years frequently traveling to China for fairly long stays. I learned enough Mandarin to get by on my own. The "scariest" thing is realizing you might have to walk for an hour in a random direction to come across a landmark like a known metro station or a hotel where you can get a taxi and have the concierge translate your desired destination.

I was mostly in first tier cities, though I did travel through some more obscure places. The worst hostility I experienced was 5 foot tall grandma with sharp elbows determined to cut in line in front of the big stupid foreigner who is passive aggressively placing his wheelie bag in her way.

If you're curious, just go. The cities are amazing, the people are friendly. Even in Beijing you can easily avoid the tourist traps. While it's not as perfectly safe as Japan or Taiwan, I spent a lot of jet lag recovery time wandering the streets late at night. Once I spent half an hour in a taxi garage at 2am at some unknown location after a 45 minute misdirected taxi ride, arranging a ride to my intended hotel. I think that's about as lost as one can get and it was fine.