I can see your point, but that (https://book.pythontips.com/en/latest/context_managers.html) requires the object you’re using to implement __enter__ and __exit__ (or, in C#, implement IDisposable (https://learn.microsoft.com/en-us/dotnet/csharp/language-ref...), in Java, implement AutoCloseable (https://docs.oracle.com/javase/tutorial/essential/exceptions...); there likely are other languages providing something similar).
Defer is more flexible/requires less boilerplate to add callsite specific handling. For an example, see https://news.ycombinator.com/item?id=46410610