We can likely use different number representations for faster results. E.g. numbers in the form of coefficients to prime factors can be multipled at O(n) time, right?
You mean like those guaranteed-always-compresses-by-at-least-one-bit algorithm patents gzip page made fun of?
In your case, doing prime factoring is where the cost would be, wouldn't it?
True but addition becomes a lot less efficient in this representation :)