The opcode is 5 maps (8, actually, but only 5 are occupied) of 10-bit opcodes, with the presence or absence of 66/F2/F3 prefixes providing 2 of those bits. If you ignore how the manual describes prefixes and look at it like that (which is suggested by the VEX encoding process), the decoding process becomes a lot simpler. In fact, with one singular exception, this is sufficient information to index into a map to figure out how long the immediate field is and whether or not ModR/M is present.