When I started dating someone whose second language was English, I found that the idioms were a giant hurdle. We really do speak in idioms and metaphors and this makes it very difficult for people who understand the words but not the cultural context. While my (now spouse) adapted, I also adapted by learning how to speak far more literally, especially around people of mixed levels of English.
I don't think there is a fear of being imprecise. If anything, it is a fear of speaking plainly as boring.
In your experience, would you say that your communication has become more precise?
Reflecting on it now, I believe that metaphors and idioms can help a lot for communicating intuitively and they can convey a lot of information. But at the same time in the situations where a idiom isn't understood by the other, I often have to take a moment to extract the essence of its meaning.