4, 6, and 8 are all fairly standard, as quadcopter, hexacopter, and octacopter, because you have a pair of motors rotating in each direction, which allows you to slow down all the motors spinning in one direction and speed up the others (as long as each set spinning the same direction is on opposite sides of the center of gravity), to maintain constant lift while yawing left or right.

Tricopters are possible as tilt-rotors, where you have two motors spinning opposite directions on mirrored sides, and the third motor is on the midline in front or back and able to tilt left and right. This allows the vehicle to control yaw despite having an inherent yaw imbalance in hover. I suppose you could do any odd number of rotors this way, and more rotors would mean less inherent yaw imbalance if there's only one extra motor spinning a given direction.

Two is possible if the payload hangs below the props, and the props are able to independently tilt.

One requires collective pitch and still at least a small tail rotor to cancel out the yaw, as a helicopter, but that is far more complex and fragile. But it is the most efficient, as fewer larger props are always more efficient than more smaller ones.

So four just turns out to be minimum to have no moving parts besides the props themselves, and still have full control authority, and the control logic is more straightforward than tilt rotors and such.

If you look at larger multi-copters, 6 and 8 are quite common since they allow for some redundancy, allowing for safe landing and/or continued flight following the failure of one of the rotors.

As other replies note, 4 is the simplest arrangement mechanically and control-wise, as the control math is quite simple (just rotor speeds/torque) and the only moving parts are the fixed-pitch rotors.

The minimum, as seen in real (and model) helicopters, is either two (approximately) constant-speed rotors with swashplate control, or one (approximately) constant-speed rotor with swashplate control and one tail rotor, either with (approximately) constant speed and variable pitch, or with variable speed. At the scale of real helicopters, two rotors may often be more powerful and efficient (e.g. CH-47, V-22) but the size and weight of the gearbox needed to transmit so much power is a significant contribution to the weight and cost of the helicopter, and thus having a single main gearbox is much lighter and cheaper. The notable difficulties of shaft drive between multi-rotor helicopters, particularly with distributed engines (see a number of V-22 issues) strongly discourages helicopters with more than 2 rotors.

Not the OP, but due to neighboring rotor interactions an even number (or a single helicopter like lift rotor and a tail rotor ignoring airfoil lift possibilities) rotor makes design easier and improves performance significantly. Think of the rotors like gears meshed together and realize that an even number spins easily, while an odd number is locked.

Of course there are 6, 8, and larger numbers of rotors used in actual drones. The advantage of more rotors is that redundancy to failure can be built in, and that rotor tip speed for a given lift can be somewhat reduced at the cost of efficiency.