I find that the feeling for me isn't about unproductive time but rather unfulfilling time.

A game can be fulfilling in some ways. Maybe it's a really good game. Or it's a way to socialize with friends or family.

It can also turn into a compulsion. That's why I avoid solo queue competitive multiplayer gaming. I try to only queue up with a friend, else I should go do something else.