I have a theory, which accounts like yours would be very interesting for.

Instead of looking at it as a per user basis, if you look at it as a network or ecosystem, the issue is that the network is being flooded with spam.

Since nothing happens all at once, over time different filters will get overwhelmed and eventually impact the less networked accounts.

It would be VERY interesting to find out when, or if ever, you begin to suspect some accounts you follow.