Imagine a graph or tree data structure where the deletion of a node causes a cascade deletion of all child nodes, which also causes a deletion of their children and so on.
This is proportional to the data structure being deleted.
Unless you use techniques to move the deletion into background threads, e.g. C++/WinRT with COM AddRef/Release, the thread will be "blocked" doing busy work cleaning all those nodes, running the cleanup code (destructors, deinit, whatever), node after node.
Yes, that's exactly what I said I understood. When Go (and Java) people say "stop the world" for GC, they mean all goroutines/threads stop, not just one. I don't think you get that with even naive reference counting.