Nah. Python gets it right; all high level languages should operate this way. Division by zero is a bug 90% of the time. Errors should never pass silently. Special cases aren't special enough to break the rules.
IEEE floats should be a base on which more reasonable math semantics are built. Saying that Python should return NaN or inf instead of throwing an error is like saying that Python should return a random value from memory or segfault when reading an out-of-bounds list index.
Exactly. Programming language design should be coherent and consistent, not just for floats, but arrays, classes, pointers, or anything else that it offers.
And the sensible thing will depend on that language.