alt Hacker NewsA xerox machine can produce verbatim copyrighted works when asked as well. That doesn’t make distributing the xerox machine the same as distributing the copyrighted works.
Strong IP advocates have argued for years that devices that can be used to infringe copyright are themselves infringement of copyright. So far that hasn’t held up to court analysis provided that device can be and is also used for non-infringing purposes. Given that so far the courts have found that training an AI model is sufficiently transformative to qualify as fair use, it doesn’t seem likely that distributing a model counts as distributing copyrighted material.
Your analogy would only be correct if I when bought a xerox machine and brought it home, I could just ask it to print out the copyrighted works without me having them to put on the glass. It's not making a copy from one I already had, but providing me a copy when I didn't have the original.
>training an AI model is sufficiently transformative to qualify as fair use
This is the key question and that courts have decided this way so far doesn't mean its the correct decision. If the model can encode the copyrighted material with sufficient fidelity to reproduce them on command, it stops being fair use or should anyway.