IANAP, but my understanding is that the Landauer limit defines the minimum energy of forcing a unknown bit into a known state. Physics as we know it is fully reversible at the microscale - every possible state have exactly one ancestor state. An irreversible process (that is, one that would force to macroscopically distinguishable states into a single one) is only possible if we conduct the "unknowness" aka entropy away from our computer - i. e. generate heat. Toffoli gate are reversible, and therefore in theory you can implement it in a way that is not subject to the Landauer limit.

Obviously, implementing one as a CMOS gate wouldn't be enough. Reversible gates would be very different. AFAIR they need to have a fan-out of one - you can't just wire an output to two inputs without losing reversibility.