I have a script that logs IPs for any traffic coming in to my servers on ports that don't accept traffic. I then block those IPs from accessing ports behind which there are services.
If they're checking my locked doors, I don't want them coming in my unlocked doors.
Closed ports are not "locked doors", and open ports are not "unlocked doors"
That is a binary thought process with a lot of assumptions. You might introduce even more attack surface in pursuit of this "security" measure by installing additional software like fail2ban, for example. Close your ports, maybe assign a non-standard port to the popular ones (like SSH) to reduce log spam, and patch your server often. Anything more complicated than that is not worth it, IMO.
That’s nice, I need to implement this.
This might be a good idea, but consider banning them for, say, a couple hours at a time. It’s easy to rotate IP, especially if you’re using a residential proxy service, and there’s a good chance you’ll end up blocking real users using the same ISP.