We do have some unique carve outs already for what we consider intellectual theft e.g. trade secrets. In this case the original artifacts might remain, but admitting the market that created them could go extinct is enough definition to be infringement at least. Fair use has been really resilient in these training cases so far but it is pretty damning to admit a negative market effect and that you're a direct substitute (see Warhol v Goldsmith recently).
I suppose it's theft in so far as you're deriving value from something that others labored for. I think that sticks in peoples throats. You can go to the library and do that already, gain knowledge, start a business or whatever. But the scale feels impersonal and monstrous in comparison.