> for RSA and ECC, is there anything preventing us from using keys 10x bigger?

you can run benchmarks yourself: openssl speed rsa1024 rsa2048

also this (slightly dated) java ex writeup covers this well: https://www.javamex.com/tutorials/cryptography/rsa_key_lengt...

tldr trade off is found between better performance and how many years the data needs to be assumed confidential