Sure, but we shouldn't stretch the analogy too far. Die rolls are discrete events, while miles driven are continuous. We expect the number of sixes we get to follow a binomial distribution, while we expect the number of accidents to follow a Poisson distribution. Either way, trying to guess the mean value of the distribution after a single incident of the event will never give you a statistically meaningful lower bound, only an upper bound.
The Poisson distribution is well approximated by the binomial distribution when n is high and p is low, which is exactly the case here. Despite the high variance in the sample mean, we can still make high-confidence statements about what range of incident rates are likely -- basically, dramatically higher rates are extremely unlikely. (Not sure, but I think it will turn out that confidence in statements about the true incident rate being lower than observed will be much lower.)