All it takes is speaking the language fluently. Hard but this mystery behind integration isn't complex. So much of the culture is expressed through language. Sound native and you will be treated as native (for better and worse)
It's rare but you will still occasionally be denied services in restaurant regardless of how well you speak the language.
Or even if you speak to people in Japanese they will try to speak to you in English.
I really believe it is not the case. It would go a long way but it is not enough. The ethnicity is also important.
There's plenty of small businesses in Japan that straight-up ask you to leave at the door if you look foreign, despite being fluent.
A lot of landlords refuse to rent to any non-Japanese people, despite being fluent.
A xenophobic society won't even let you participate fully, much less integrate, no matter how well you speak the language.