Big Tech backs AI training opt-out before IETF consensus

Google, OpenAI, Microsoft, and Anthropic publicly lined up behind a single robots.txt token for opting out of AI training days after Kevin Kelley of Anthropic posted it as an individual Internet-Draft, not as a product of the IETF AI Preferences working group. Site operators would name a "policy token" on an ordinary User-Agent line, such as __usage_ai-training, and use existing Allow and Disallow rules to say where training is preferred and where it is not. The signal is meant to govern use, not fetching: a crawler's own product token would still control access, and files that never use the token would express no preference at all.

The working group had parked attachment, the question of how preferences are written into robots.txt, so it could finish a shared vocabulary first. Kelley wrote that he limited the draft to training because that definition had stabilized, and that he was describing what he was announcing rather than replacing the group's work. He also said attachment had been on hold and that he treated it as the more mechanical half of the problem.

That reading did not settle the list. Laurent Le Meur of the European Digital Reading Lab called the company statements a fait accompli and asked whether the token would also stop indexing inside search systems that train models. Nate Hake, who runs TravelLemming, said creator circles read the announcements as a Big Tech opt-out scheme for copyright, covering training and not inference, and that he had been doing damage control over his own participation. Alissa Cooper noted that the draft is silent on how other standardized tokens would be defined, with no registry.

Chair Mark Nottingham said the group is unlikely to treat a training-only profile as a replacement for its attachment work, and that any decision would still go through consensus. He welcomed early implementation as a source of running code, but warned that early deployers own the interoperability risk: the working group's term is train-ai, not the token the companies are already using, and the two are close enough to confuse site operators.

Separately, Alex Gibson of News Corp demanded that he and Max Gendler be removed from the draft's acknowledgements. The text thanked them, with their News Corp affiliation, for shaping discussion and for a "partnership." Gibson wrote that neither contributed to the draft and that News Corp does not support it. Kelley said the list was meant to credit working-group participation, not endorsement, and that he would make that explicit or drop the names once the Datatracker lock lifts.