>Anecdotally speaking, I've personally lived in multiple countries on multi-year-long temporary resident visas without the right to work, because getting the right to work is a separate process from being allowed to be somewhere. So have family members. And in case it makes a difference, which it might, one of those countries was the US.

Usually those long term visas (eg. student visas, I guess) require some sort of extra vetting to acquire (ie. they're not the tourist visas that get auto-issued at the border), and therefore are unlikely to be possessed by "people who aren't supposed to be here", so as a practical matter I agree they should be accepted in place of a "work visa".