Yeah, it's important that your own people are doing regular inspections of the production line, keys get burned at the final station (use temp keys for QA stages before that point), and you retain software control of the HSM.

You probably aren't going to do this whole rigmarole for small runs of devices like in the OP.