Thanks, though I realize (thanks to tptacek) that I didn't properly emphasize that the determinant must be constant but non-zero.

If the determinant were zero, then (for basically the same reason as why you can't divide by zero) the matrix wouldn't be invertible. You wouldn't expect the polynomial to be invertible in that case, either.

If the determinant had free variables left over, then there is some combination of variables that will make it zero (e.g., if it came out to x^2-1, then it would be 0 for x=1 or -1). So maybe inversion won't fail everywhere, but it will fail in those spots, which would also lead one to think the polynomial is not invertible.

But if it's constant and non-zero, then there is no place where you can't invert the Jacobian. So it seems very plausible that this would apply to the polynomial as well. That's essentially what motivated the conjecture in the first place (leaving out some details, of course).