> Downloading every torrent of all e-books ever is pretty clearly illegal (or at least it fuckin would be if I did it). Pretty sure like, multiple labs have been popped for that though.

Oh it is, and at least anthropic has paid $1.5 billion and deleted there torrented copies and not released any models derived from them as a consequence.

The thing is it turns out to be not that expensive to just buy a copy of every book legally and scan them. And there's even precedent that this is legal predating LLMs (Google books)