To be fair, Musk was only parroting what Karpathy was telling him so you should ask him how self driving cars are supposed to work with CV only.