This is one of my favorite things with the Mac. I feel it is very intuitive me moving the app to Applications, then just move it to trash if it sucks and it is completely gone.
setup.exe .pkg installers make me feel dirty because I hav enough control over how it will fuck up my system or not.
I think most of the complexity is around the concept of disk images. If the app was just distributed in a zip file (like many in fact are) then you just unzip it, and the application is right there. You can move it to /Applications if you want (and many apps offer this) but it's not necessary.
Disk images seem to be much rarer on other platforms, and are usually only used for images of optical disks. Even the concept of "ejection" can be confusing, especially since when mounted disk images appear almost identical to actual physical removable media. I'd like to hear the reasoning behind why mounting disk images isn't relegated to a vestigal "power user" feature by now, since it seems like zip files (especially in appledouble format) can serve the same needs for most normal user flows.
The app icon approach only give the illusion of control. That may be worth something too, but whether you run a .pkg or a program, the app developer can do whatever they want with the permissions you grant it.
There is an advantage to being able to move a program to any locations and run it from there, and while the same can easily be done on Windows (just make one big .exe!), it's not expected for users to just move install directories around.
That said, programs break the same way: configuration pointing at wrong directories, permissions going wacky, you name it. It just seems to go wrong less often on macOS because of developer expectations.