The issue is if rare is simply "not many in circulation", that's very very different from rare being "it is a unique artifact". The former kind of rarity doesn't really present an existential end, whereas the latter does.
Yes this is disingenuous to the extreme.
My neighbor self-published a book, printed I think 100 copies at his own expense. It's literally a rare book. I can't imagine he nor anyone would care if an AI company bought a copy, no matter what they did with it.
Maybe these "rare books" are first edition Mark Twains, and. maybe they're unwanted books that would otherwise have gone to be pulped. The distinction is important and by not giving any evidence or even a qualitative claim about the types of books, 404 media is being pretty weak here.
I don't think HN commenters need to be the arbiters of which books are beneficial to society.
And beyond just benefit to society, don't forget how subjective the decision about what constitutes a "unique artifact" is. That random product manual from 1966 becomes a near-priceless relic to me if it lets me repair and use the sewing machine handed down from my grandmother. Not necessarily the ink printed on paper, but the information contained within.
The above is based on a true story involving archive.org's Manual Library. The idea that even such esoteric and forgotten information would get hoovered up into the walled garden of Amazon's AI and then destroyed in the outside world, such that I have to go pay them to access even a facsimile of it, is frankly disgusting.