It's good to hear I'm not the only one with the eye contact thing — I also noticed about myself that if I'm looking in another person's eyes, it means their words aren't really registering in my head.
This also came up when trying EMDR in a mental health setting, where you're supposed to follow the therapist's pointer with your eyes while thinking about past events. I found that I couldn't think about anything else except making sure my eyes were following the pointer; directing my gaze took up all the available mental bandwidth.
A bit of a tangent. As an extrovert and someone who is known for being very easy to talk to as well as conversational, I also do not look directly in a person's eye. I definitely face the person and look at their face. But not directly in their eyes.
I know this because years ago I saw an interview with Ken Griffin (hedge fund billionaire). He does a lot of interview and makes intense, enduring eye contact[0]. Juxtapose that to someone like Jaime Diamond who also gives a good many interviews but takes breaks/pauses to look away.
So after seeing the video, I tried making conversation with people while looking directly at one eye or the other. And at first I found myself paralyzed and barely able to keep focus on my train of thought. Eventually I was able to get a bit more comfortable doing it, but it required a great deal of effort. And anecdotally I found that the people I spoke to were more likely to look away the longer I kept the eye contact.
[0] - https://www.youtube.com/shorts/d4HaX2IUFJc