We Tracked a Shipment of Rare Books. It Ended at an Amazon AI Training Facility

The length of this article and the unnecessary visualizations felt like a waste after getting to the end and discovering they won’t reveal anything about the books that were scanned.

The closest they got was admitting that the rare books weren’t anything that someone might care about in the sense that people assume when we hear “rare books”

> As the bookseller who sold them told me, there are not many people in the world who would care about them in the same way people might care about the first edition of Oliver Twist, but that doesn’t mean they’re not valuable.

Okay? But then why exactly where they considered valuable enough to warrant an entire article about them going to a book scanning facility? Without revealing anything about these books I have no idea if they were classic literary works that were underappreciated, or if this was some old guide about How to Use Microsoft Office 97.

There was a more balanced take on Twitter (which I’m unable to find again, because Twitter) from a book seller who said it was more of the latter type: Books that were rare because they were no longer in demand and most everyone had thrown their copies away. Some parts of the media are doing backflips to try to imply that these are cherished literary classics being fed into the shredder to deprive humanity of something valuable, but the book seller seemed happy to be making sales for useless old books that no human was interested in buying.

> We’re not revealing the titles of the books included in the shipment we tracked, but they are rare, meaning there are not many copies of them in circulation. Sometimes that’s because not many copies of them were ever printed, and sometimes because they are in a foreign language not many people speak.

Not even the title of one of those rare books?

Why is the textual data bound up in these paper books worth the trouble?

These LLM training runs have already ingested essentially the whole public internet. What marginal value is to be gained from scanning and destroying obscure books?

For me the biggest functional issues with LLMs don't seem to have any connection with "I wish they had read this obscure community cookbook from 1946". Is that going to get Claude to stop saying "honestly" to me? Is it going to get LLMs to stop making up sources that don't exist? What is in it for Amazon or any LLM company to chase more obscure data like this.

I don't understand why Amazon still bothers to blow money on their in-house AI effort (outside AWS Bedrock). What have all the scraping, scanning and training led to? Nobody is using their models.

Great article and work to reverse engineer who was mass buying used books.

I watched Short Circuit yesterday. All I can think now is "need input".

According to the article, the scanning facility has been there for a while, so it may well be that they have been destructively scanning books for a while, even before AI. It makes sense for the major book retailer to have digital copies of every book so that it can begin selling those as soon as it can negotiate the rights (if there are any).

In any case, all the indignation about destructive digitization misses the point that rare books takings space in a warehouse for years without being bought will eventually be destroyed anyway.

I look forward to the day that AI companies pay large advances for new books.

It'll take a while before publishing collapses due to the availability of the same information via an LLM, but once this happens new books will get a lot more expensive for AI companies and a lot cheaper for everyone else.

Does anyone else get pretty tired with 404s style of writing? I hate to make this comparison but it feels like a Fox News for the left.

I always struggle with their articles because it feels like they built it for rage bait on a topic and leave the other interesting topics out of it. It is an absolute shame for books to be destroyed but what does it really mean to be rare here? I know they kind of tried to differentiate but it sounds like this could be John Doe’s self help book that never sold well. If you ever are connected to a library you will start to realize how many books simply get thrown out or sold for nothing because nobody wants them.

To me the problem is partly the copyright law. I think it’s. Hard problem but I always lean more towards books having short copyright shelf lives and making it legal for digital copies to be shared after which I think would eliminate I good part of this problem. Not to mention 99% of the books published are probably garbage but that is highly subjective.

So it is bit of a meta rant but I think there a couple holes to go down that could be extremely interesting but they always write these informationally light articles. Like scrolling through a NYT visualization for just some shipping datapoints. Don’t really dig deep on anything and then end with a trust me bro these are rare books that Amazon is destroying for AI when I cannot be that upset with the amazons of the world. There are a lot of reasons for a business to digitize books, most books are worthless and it makes sense to cut the bindings for scanning. I would rather talk about how could you fix copyright to make this less an issue but is it even an issue with how many books get thrown out?

Discussions:

2 days ago https://news.ycombinator.com/item?id=49310725

21 days ago https://news.ycombinator.com/item?id=49068738

It seems like most bookshop owners today are hoarders-turned-entrepreneur. They rent a storefront, adopt 2-5 cats, and they set up shop with their hoard of books so that they can hoard on a commercial level that would never be possible in a private home.

It also seems that American society has been swept away by the worship of books, rather than literary appreciation or promotion of education. "Banned Books Week" as the primary exhibit here. A tug-of-war over books that are supposedly "banned" when the verb itself has been twisted beyond recognition. I see posters and television shows and public service announcements that promote "Books!" and "Reading!" for no other purpose. Many books are trash and they will fill your head with garbage as sure as social media can, so why the indiscriminate worship of books?

It is conspicuous that, aside from established booksellers, and perhaps the LFL owners, many of the people crying out and moaning over book-destruction seem unwilling or unable to actually take those books in and store them. That is the point, that these books are unworthy of taking up storage space, which is an ongoing cost, and maintenance, and therefore, it is fiscally responsible to sell them, scan them, and destroy them. If you want to dedicate a room full of shelves to books that nobody will ever read, then go ahead and buy out your local bookseller! They will not stop you or cancel your order! Tell them you are saving those innocent books from the Big AI Bugaboo! They'll grovel at your feet!

Also now, I'm seeing small booksellers who feel "suspicious" or "skeptical" about large book orders. There was one Down Under who said "oh we got an order for over 70 and we can't physically handle that!" I feel like it's becoming a "shut up and take their money" situation. These booksellers, as professional business owners, should know that they may never liquidate stock, and this is their golden opportunity to simply get rid of some of that excess hoarded inventory.

But if you're merely cosplaying as a bookseller, and deep down you're a hoarder and a worshipper of books, you may really be reluctant to do transactions or earn money that sells away your books.

Something I've been thinking about is that a physical book is a physical artifact, and it's age and authenticity can be verified by treating it as such.

In contrast a digital book has no such marks. It's impossible to tell if it was redacted to hide inconvenient passages, or even completely rewritten or fabricated (which can now easily be done at scale with AI). If the physical book is a primary source then the digital version must be considered a secondary source: A potentially biased retelling.

For this reason, a digital book can never be a perfect substitute for the physical book it was created from.

if we are gonna pearl clutch about this problem then the headline should be about how libraries -- the entities who preserve rare books -- don't have enough funding to do this. why are we now expecting booksellers to preserve rare books? they are booksellers, in trade

Bad enough to steal IP but destroying books…that’s just evil

How many AI companies are doing this? Say there are only 5 copies of a book left out there but 100 different companies want to shred it. Rare items could effectively vanish from the market. "Rare" meaning inclusive of high-quality items, disregard the bot opinion that mass book destruction is fine because most books are worthless anyway. And are any safeguards in place to make sure scanned books get saved in their original form for posterity before being recycled into trainingslop? Aside from Google's quasi-legal/ethical mass book scanning operation of course.