Doesn't the conversation also happen WITH words? Like Chuckie tells him quite directly in one of the best scenes in the film: "I mean, you're sittin' on a winnin' lottery ticket. And you're too much of a pussy to cash it in, and that's bullshit. 'Cause I'd do fuckin' anything to have what you got. So would any of these fuckin' guys. It'd be an insult to us if you're still here in 20 years. Hangin' around here is a fuckin' waste of your time."
Chuckie is the one who can say that so plainly to his face, too. A math professor he’s only getting to know saying plainly that his life to this point is inferior, that he should leave all his friends and compatriots behind, and that he’s wasting his time with the people he loves is going to come off as condescending and arrogant.
Showing him his potential and telling him he can do great things is awesome. Crapping all over him for not having the background of the average student would just push him away.