A frontal cortex isn't enough; there are plenty of corner cases that humans fail at too. The real test is if self-driving performs on par, or better than, humans in the vast majority of cases. If it saves 50,000 lives a year to go with self-driving, it's a net-win even if there are a few people who die in situations where they would have survived with a human driver behind the wheel.
Self driving cars are not going to be accepted if they have only marginally better success rates than humans. Just look at the news. Every minor self driving incident is endlessly magnified by the media while millions of human-caused accidents are just a part of life. That's just how our brains work. All major decisions are made primarily based on emotion, not analytics.