Interestingly it fails for me because my `su` isn't world-readable:
$ stat /bin/su
File: /bin/su
Size: 59552 Blocks: 118 IO Block: 59904 regular file
Device: 0,52 Inode: 796854 Links: 1
Access: (4711/-rws--x--x) Uid: ( 0/ root) Gid: ( 0/ root)
Access: 2023-09-18 13:23:03.117105665 -0500
Modify: 2021-02-13 05:15:56.000000000 -0600
Change: 2023-09-18 13:23:03.119105665 -0500
Birth: 2023-09-18 13:23:03.117105665 -0500
I'm not sure I have any setuid/setgid binaries that are world-readable...It being readable is the default configuration most places, after all the purpose is to call it from a non-privileged user. But I could see it being made non-readable since its use is discouraged nowadays... though then I'd expect sudo to be readable as an alternative.
A workaround might be to make all setuid/setgid files non-world-readable because then they cannot be opened at all, and thus there is no setuid file to replace the contents of.