I only learned about aphantasia a few years back and, like you, so many things in my life just clicked when I realised I was an outlier.

I could never relate with people when we'd read a book that had become a movie and they're like, "but they're not at all how I imagined them" and all I could think was "but the person in the movie has the exact characteristics described in the book!"

When people would ask me to imagine something then ask questions about it, I'd basically just be deciding details on the fly and then retaining them in a list. For example, if you said, "think of a car," inside my mind is just "a car" on repeat. Then if you asked me what colour it was, I might decide "red" and my mind is now, "a red car", etc.