It is impractical to try to recover/continue your program in an out of memory situation for most programs.
So it is a good advice for beginners.
If the code is in a library (and I’d treat code as if being part of a library by default), then the library shouldn’t be deciding that.
But in order to able to show proper diagnostics and error messages, you should still return all the way to the top.
I think this is fair the majority of time, but I'd like to mention Zig, since Zig has a convention of all allocations being fallible and handled. It's a pain at first to handle error.OutOfMemory at each allocating site, but I feel like I'm much more conscious of where allocation can fail and how to gracefully handle it. I've also gotten a lot better at transactions since pretty much every operation has failure points now.
In fact I've written a whole interpreter that can recover from OOM by raising a recoverable exception to the user. It's really only because Zig made recovering idiomatic, and I'm not sure I could've done it in another language (maybe Rust but I'd have to rewrite large parts of stdlib to both return an error and take a custom allocator).