Is there a way to do that and still be able to access the server?
Yes, cloudflare tunnels do this, but I don't think it's really necessary for this.
I use them for self-hosting.
Yes, CloudFlare ZeroTrust. It's entirely free, I use it for loads of containers on multiple hosts and it works perfectly.
Many ways. Using a "bastion host" is one option, with something like wireguard or tinc. Tailscale and similar services are another option. Tor is yet another option.
Either via a VPN or a tunnel.
Yes, of course.
Free way - sign up for a cloudflare account. Use the DNS on cloudflare, they wil put their public ip in front of your www.
Level 2 is install the cloudflare tunnel software on your server and you never need to use the public IP.
Backend access securely? Install Tailscale or headscale.
This should cover most web hosting scenarios. If there's additional ports or services, tools like nginx proxy manager (web based) or others can help. Some people put them on a dedicated VPS as a jump machine.
This way using the Public IP can almost be optional and locked down if needed. This is all before running a firewall on it.