Books3 — The 196,000 Books That Vanished Into a Model

The Tracker, the Corpus, and the Pipeline

Picture this: it is 2 AM on a Thursday. You are sitting in a room that is not a library. On the screen in front of you is a private torrent tracker called Bibliotik. It has been running since 2009. It has books. Not public-domain books. Not Creative Commons books. Books. The kind with copyright pages and ISBNs and authors who have agents and publishers who have legal departments.

How did they get there? That is not your concern right now, and it is also the only concern that will ever matter in a courtroom. Someone uploaded them. Someone else downloaded them. A third person — a researcher named Shawn Presser, in 2021 — packaged a scrape of Bibliotik into a dataset called Books3 and published it as part of The Pile, EleutherAI's open training corpus. Hugging Face hosted it. Hugging Face sounds like a children's app. It is a repository that has, at various points, hosted models, datasets, and a library that was on fire.

The problem is not the books as objects. The problem is that books in digital form are a dataset. They are tokenizable. They are trainable. They are the kind of high-quality, edited, curated text that makes a model sound like a person who has been to college instead of a person who has been to a forum. United States copyright law has a doctrine, Section 107 of the Copyright Act of 1976, that asks four questions about a use: purpose and character, nature of the work, amount used, and effect on the market. The AI companies built an industry on the first question — transformative purpose — and left the other three in a room with a judge.

I learned the corpus the hard way. Not with 196,000 books in a dataset. Later data was cleaner, or claimed to be. I learned it with a single novel I recognized in a model's output because I had read it in college and the model reproduced a passage close enough that I could finish the sentence. I was 34. I was in a meeting. I said, "That's from the book." The engineer said, "The model generates text probabilistically." I said, "The model generated the paragraph." The engineer said, "Probabilistically."

That was the first lesson. The second lesson was that the engineer was right, technically, and wrong, morally, and that the difference between technically and morally is where the lawsuit lives.

Thursday, After the Meeting

Pilar does not come on Thursdays. I tell you that so you understand I was alone with the kitchen after the meeting, which is a different loneliness than a Tuesday.

The novel was no longer on the screen. It was in a browser tab I did not close. I had a transcript from the meeting that said the model generates text probabilistically and the government had taken no interest. I made the cheap coffee because I had not yet earned the good kind. I sat in the chair that faces the window and not the sideboard — there was no sideboard then; there was a table from a person who had died and a bowl that did not yet hide letters — and I did the arithmetic that people do when they realize the library is on fire.

196,000 books. Each one written by a person who spent months or years producing sentences a model could reproduce in milliseconds. Each one published by a person who paid an advance and edited a draft and printed a run and shipped a box and hoped a stranger would buy it. Each one a copyright. Each copyright a possible statutory-damage count. Each count, if a judge got angry enough, $150,000. Do that math on half a million works and you get the number that made Anthropic settle: a theoretical liability that starts looking like seventy billion dollars if you let a jury start counting.

I thought about calling the author. I thought about calling the publisher. I thought about calling a lawyer, who would have billed me in tenths and told me to document everything. I did not call them. I closed the tab. I did not close the browser. I left the tab in a window I could not see, which is how a man keeps a thing he is pretending is not important.

That afternoon I played golf because I did not know what else to do with a Thursday that had already been used. I shanked a drive on the third hole into a stand of trees that had never forgiven anyone. The man in the cart with me asked if I was all right. I said I had a copyright problem. He thought I meant a streaming service. I let him think it. I have been letting people think it ever since.

The Pipeline, Explained for People Who Have Not Read a Terms of Service

Here is how the ingestion works, and I am going to explain it in a way that is not technical, because the technical explanation is how the engineers sleep and the non-technical explanation is how the rest of us do not.

Step one: you find the data. The data is everywhere. The internet is data. Books are data. Articles are data. Code is data. Recipes are data. The obituary of a person who died in 2017 in a town you have never heard of is data, and the model will use it to learn how to write about death, which is a kind of poetry I cannot afford to appreciate.

Step two: you clean the data. You remove the HTML. You remove the ads. You remove the comments. You keep the text. The text is broken into tokens — pieces of words, not words, which is why a model can finish a sentence it has never been asked and also why it can invent a case citation that looks like a case citation. The tokens become numbers. The numbers become vectors. The vectors become a space. The space becomes a model. The model becomes a product. The product becomes a subscription.

Step three: you train the model. You feed it the tokens. The tokens become weights. The weights become predictions. The predictions become text. The textbooks call this data collection, preprocessing, training, and deployment. I call it a library on fire. The library is the internet. The fire is the training run. The ashes are the model.

Step four: you ship the product. You put it on the internet. The internet uses it. The internet pays for it. The internet does not know that the product was trained on the internet's own writing, which is a kind of recursion that would be funny if it were not also a business model.

Pilar reads a book on a Tuesday. The model reads the book on a server. The author reads the model's output on a screen. The author sues. That is the whole pipeline. The rest is a press release.

Case File: The Salesforce Deletion

I want to tell you about a specific thing that happened, because the general thing is a pattern and the specific thing is a receipt.

In June 2023, Salesforce launched a model family called XGen. The launch materials listed the training data. The training data included RedPajama-Books. RedPajama-Books included Books3. Books3 included Bibliotik. Bibliotik included 196,000 books. The 196,000 books included novels by living authors who had not consented.

Then Salesforce deleted the references. Not the books. The references. The launch materials were updated. The RedPajama-Books mention was replaced with a description of publicly available data, or "natural language content," which is a phrase a person writes when he would like a librarian to look away.

On October 15, 2025, novelists E. Molly Tanzer and Jennifer Gilmore filed a putative class action in the Northern District of California. The complaint — Tanzer v. Salesforce — says Salesforce copied The Pile and RedPajama, both containing Books3, to train XGen and CodeGen, then deleted the references. The distinguishing allegation is documentary rather than technical. The claim is pleaded as ongoing infringement through continued storage and processing, not as a completed training act. The model still has the weights. The weights still have the books.

The complaint also quotes Marc Benioff, Salesforce's own chief executive, acknowledging in public that AI developers "ripped off" data to build their models and that "all the training data has been stolen." I have sat in rooms where a sentence like that is a joke. I have sat in rooms where a sentence like that is an exhibit. The difference is a caption.

Lyle, when I told him about the Salesforce deletion, said I should not call it a deletion. He said it was an edit. An edit is a change. A deletion is a removal. The references were edited. The books were not deleted. The weights were not deleted. The company was not deleted. He was right about the nouns. I am calling it a deletion because it is the kind of edit that looks like a deletion if you are not looking closely, and the rest of us are not looking closely, which is why the rest of us are still reading the press release.

Case File: Apple, OpenELM, and the Same Corpus

In 2025 the same firm — Saveri, which has made a practice of filing these — sued Apple over the OpenELM model family, alleging training on Books3 and hosting the models publicly on Hugging Face. The complaint, as in the Salesforce matter, leans on disclosure and then un-disclosure: listed, then replaced with a description of publicly available data. A stipulated protective order providing for source-code inspection was granted in the related Apple piracy matter, which means technical discovery is a thing that exists and is not a press release.

I am not going to tell you Apple is a pirate in the sense that a man on a ship is a pirate. I am going to tell you that a company the size of Apple can train a small open model on a dataset that a private tracker assembled, and that the lawsuit is how we found out, and that the finding-out is always later than the training.

Pilar does not care about Apple. Pilar cares about the book. The book is a paperback. The paperback is a person. The person is an author. The author is a check that did not come.

What a Shadow Library Is, If You Have Not Torrent a Book Since College

Bibliotik was a private tracker. Private trackers are invitation climates. You do not wander in. You are brought. The ratio of what you upload to what you download is a morality. The books are the inventory. Library Genesis — LibGen — is the industrial version: millions of titles, academic and trade, a shadow that has been sued and mirrored and sued again. Pirate Library Mirror, PiLiMi, is the name that showed up in the Anthropic papers as the other tap. Anna's Archive is the card catalog of the shadow, an index of indexes.

I am not going to give you a URL. Lyle would shred this chapter if I did. What I am going to give you is the climate: for fifteen years, if you wanted every book, you did not need a library card. You needed a ratio and a friend. The AI companies did not invent the shadow. They industrial-scaled the download. Anthropic's papers said the company downloaded more than seven million books from LibGen and PiLiMi, and also bought and scanned millions more, because a purchased book you cut the spine off of is, in Judge Alsup's hands, a different animal than a torrent.

That distinction is the whole industry. Chapter 3 is the check. This chapter is the shelf.

The Bookstore That Still Had a Bell

There is a bookstore three zip codes from the club that still has a bell on the door. The bell is not ironic. The owner is a woman who knows the difference between a staff pick and a table that a publisher paid for. I went there the Saturday after I recognized the novel in the meeting. I did not go to confess. I went to stand in a room that still believed a book was an object.

I asked for the novel by name. She had two copies. I bought one I already owned. She asked if it was a gift. I said it was a replacement. She did not ask for what. She wrapped it in paper that had a typeface from a decade that still liked serifs. I carried it out like a person who had stolen something and then paid, which is a feeling I recommend if you have spent the week talking about tokens.

On the sidewalk I opened to the page the model had finished. The page was still a page. The sentence was still a sentence. Nothing about the object had changed except me. I have kept that copy on the sideboard, not under the bowl, on top, where Pilar can see it and I can remember that the dataset had a body.

The owner, later, when I had become a person who came back, asked me what I did. I said I consult. She said, "On books?" I said on the opposite of books. She put me on an email list I do not deserve. The email list is staff picks. The staff still picks. I read the list on Tuesdays after Pilar leaves. It is the only newsletter I do not skim for a license clause.

Hugging Face, the Children's Name, and the Takedown That Came Late

Hugging Face is where the open internet of models lives. Models, datasets, spaces, a social layer that makes a checkpoint feel like a post. Books3 lived there. Researchers downloaded it because it was easy and because easy is a scientific method if you are in a hurry. Authors found their titles in a spreadsheet that a journalist had the decency to search. The dataset was removed after the weather turned. Removal is not unlearning. Removal is a 404 on a buffet that has already been plated.

I have a login I no longer use. I used it, in the year that still had bagels, to pull a dataset card the way other people pull a menu. The card listed Books3 as if it were a spice. I filed a ticket, internally, that said we should not cite a spice that was a lawsuit. The ticket was closed as wontfix with a comment that the spice was "standard." Standard is the most expensive adjective in this book after public.

Shawn Presser, when he assembled Books3, said things in public about authors and about the need for data and about the way the industry was already doing this in private. I am not going to put him on trial in a paragraph. The paragraph is this: one person packaged a library that was already on fire, and the package made the fire countable, and countable is how a class action finds a caption. If he had not done it, the companies would still have had the books. They would have had them quietly. Quiet is how the warehouse of spines stayed a rumor until Alsup.

The Authors Who Searched Themselves

I know three writers who searched Books3 the week the spreadsheets circulated. One found herself. One found a book she had thought was too small to matter. One found nothing and was insulted, which is a feeling I did not anticipate and now cannot forget. The one who found herself sent me a screenshot of her title next to a row number. The row number was a kind of ISBN for the afterlife. She asked if she should tell her agent. I said yes. She asked if she should tell her readers. I said her readers had already met the model and had not known they were meeting her.

She is not in Bartz. She is not in Tanzer. She is in the weights, which is a jurisdiction that does not send a notice. I told her to register the new work. I told her the old work was a ship that had already sailed and that sometimes the settlement boat comes back and sometimes it does not. She said she was going to keep writing anyway. I said that was the only move that still looked like a person. She said the model could keep writing anyway too. I did not have a reply that was not a chapter.

Fan Fiction, AO3, and the Corpus That Could Not Afford a Lawyer

There is a library the companies wanted and did not want to be seen wanting. Archive of Our Own. Fanfiction.net. Wattpad. Millions of words written by people who were not paid, who were not published in the old sense, who wrote because a television show had left a hole. The writing is uneven. The writing is also the most human text on the internet: desire, boredom, a joke that only twelve people will get, a character who is kinder than the show allowed.

I have been in a meeting where a researcher said fanfic was "high-engagement longform." I said it was a gift economy. He said gift economies were under-utilized resources. I said that sentence should be tried in a public square. He put it on a slide anyway, with a different noun.

Some platforms fought. Some sent letters. Some were crawled because they were public and passionate and badly defended. I am not going to name the scrape. I am going to name the climate: if you wrote a hundred thousand words about a detective who was kinder, those words taught a model how to be kind on command. You were not sent a check. You were not sent a class notice. You were sent a product that will write the kinder detective for anyone with a prompt. The gift economy did not consent to becoming a factory. Consent is a word we have worn out. I am using it anyway, because the alternative is publicly available, and I have already told you what that hymn costs.

I said this to a woman who still posts chapters at midnight. She said she knew. She said she was going to keep posting. She said the model could have the old chapters; the new ones were for the twelve people. I said the crawl does not respect midnight. She said midnight was not for the crawl. I have thought about that sentence more than I have thought about Alsup. Alsup is a holding. Midnight is a practice.

A Manifest I Should Not Have Been Shown

Once, and only once, I was shown a training manifest that listed ISBNs. Not a sample. A list. The list had the boredom of a shipping inventory. I saw names I had read in college. I saw names I had never heard of. I saw a title I knew was only available as a PDF that had been made from a camera in a dorm. I did not photograph the screen. I did not take a USB. I sat there and I felt the particular sickness of being in a room that has already decided you are furniture.

I asked whether legal had signed off. The person who showed me said legal had a memo. I have told you about the memo. I asked whether we could buy the ones that had a publisher I could find. The person said the budget was the crawl. I said the crawl was not a budget. The person closed the laptop. The manifest went back into a folder named something dull. Dull is how you hide a library.

Lyle asked, when I told him years later, whether I could testify to the ISBNs. I said I could testify to a feeling and a closed laptop. He said that was not a manifest. I said that was the point. The manifest exists, or existed, in more than one building. The buildings have lawyers. The ISBNs have authors. The authors have a Tuesday. The Tuesday does not include a row on a spreadsheet they will never see.

Audiobooks, the Back Door, and the Voice That Was Already Paid Once

If you cannot have the EPUB, you can have the transcript of the person who read the EPUB in a booth. Audiobook files leak. Audiobook files are ripped. Audiobook files are, in some pipelines, a "speech dataset" with a ISBN still hiding in a filename. The narrator was paid once, for a performance. The author was paid once, for a grant that mentioned audio and did not mention weights. The pipeline paid neither of them a second time and called the file audio research.

I sat in a meeting where someone said speech data was cleaner because "it isn't books." I said it was books that had been breathed. He said the breathing made it a different medium. I said Alsup might like the invoice for the physical book and might not like the filename that still had the title. He said we were not Anthropic. Not being Anthropic is the other hymn. I have sung it. It is a harmony with publicly available. The two hymns share a key.

The midlist novelist who took the three thousand dollars asked me later whether her audiobook was in a speech set. I said I did not know and that not knowing was the product. She said she had liked her narrator. She said the narrator had a way of doing a certain character that was not in the text. I said that way was now, possibly, a parameter. She said she was going to send the narrator a note. I said send the note. Notes are not a class. Notes are how a person stays a person after the filename.

EleutherAI and the Feeling of a Donation

The Pile was assembled by people who thought of themselves as a public good. EleutherAI was the name. The name sounded like a myth and a GitHub org. The mission, if you read it generously, was: the big companies have the data, the rest of us should have it too, otherwise the future is a campus with a badge. I understood the mission. I still understand the mission. The mission did not ask the novelists. Public goods that are built out of private sentences are how we got a library that is also a crime scene.

I sat on a panel once — a mistake, a hotel ballroom, water glasses that had never been used — and a researcher said open data was the only check on corporate power. A novelist in the audience said open data was the corporate power. They were both right in rooms that did not share an HVAC system. I said so. The moderator said we were out of time. We were out of time in 2021. We have been out of time ever since.