Applying the ? operator to an Option<_> doesn't work inside a function returning io::Result<_>, for example, and turning the None from an out-of-bounds vector access into some io::Result::Err usually wouldn't make sense.

I would go so far as to say that an out-of-bounds index is always a logic error (to be handled by panicking), not an input error (to be handled with Result). And if the out-of-bounds access was caused by invalid input, then logic error consists of not checking the input before indexing.