It's a mix of things. Japan's reputation for extreme cleanliness was partly engineered in preparation for the 1964 Olympics, in which organizers spent ~5% of the budget on beautification efforts. Arguably they started out from a higher personal baseline with bathing culture and there's also issues of population density, monoculture, and social sanctions. But there's also something of a gap between trope and reality; public urination is pretty common in Japan, especially at night, which shocks some people.