That’s why most doors come prehung in a jamb. Just add shims and then cover them with trim.