That's what I said. If you're willing to implicitly rely on webmail provider's identity/auth systems then you can simplify - but that's not much different to just adding a Sign in with Google/Microsoft button. The identity is ultimately controlled by the email service and almost all users are on just two of those.
What web auth system doesn't rely on a user having a secure inbox? It is vastly different from signing in with an IdP, since I can use my email address regardless of who my mail provider is.