The problem with this is, you spend a lot of effort for low benefit. You should spend it on actual security instead.
What would be "actual security" in this context?
This isn't about security of the same kind as authentication/encryption etc where security by obscurity is a bad idea. This is an effort where obscurity is almost the only idea there is, and where even a marginal increase in difficulty for tampering/inspecting/exploiting is well worth it.
Changing a port and enabling aslr are not "a lot of effort".