Even if you don't have Forward Secrecy, like you decided to use RSA KEX which is a terrible non-default idea even in 2015 let alone today (this feature isn't even present in TLS 1.3 deliberately, lobbying to keep doing this failed), your private key is still needed so a third party CA can't imitate you.

The CAs have never been supposed to know your private key. For a long time now it's straight up forbidden on pain of removal from trust stores for the CAs to learn somebody else's private keys.

For the example of Let's Encrypt your client probably picks a private key and stores it where your web server can use it, but it never sends this key to anybody else. In fact if you care you can even have the key chosen by the web server and literally never send that key to the Let's Encrypt client at all, the client picks up a "Certificate Signing Request" and it goes OK, I see you want a certificate for some key you know but I don't, that's cool I will go ask Let's Encrypt to issue a certificate for that and let you know.