This is the only way I could come up with that would allow an end user to do a full factory reset, and end up back in a known good secure state afterwards.

Storing it in the firmware would mean every user has the same key. Storing it in eeprom means a factory reset will clear it. This allows me to ship hardware with the default key on a sticker on the side, and let's a non technical user reset it back to that if they need to.

It gives you a 256bit block to work with - https://docs.espressif.com/projects/esp-idf/en/stable/esp32/...