As far as I understand it, they aren't being allowed to drive. They are doing the equivalent of "ignore that, it's not a real obstacle" or "try to go around this way", and then the car takes that input into account and does the actual driving (steering, control of throttle/brake) on it's own as usual.