So my PC runs 5% slower because someone could break into my house to get physical access to decrypt memory? OK sure, but not my top concern, and a bad tradeoff for the lost performance. And not only fair, but completely accurate to describe TSME as non-critical for *most* consumer desktops. I'd go as far as to say useless and counter-productive for most, but not all, consumer desktops.
Does it run slower? I'd expect dedicated hardware to do that encryption/decryption, in which case there should be no difference.
So you turn it off by default in BIOS and allow those that feel it's useful to them to enable it, and you solve for both sides of the problem.