Making it easier for websites to express AI training preferences

OpenAI announced it will honor a common signal in robots.txt that allows websites to express their preferences regarding the use of their content for foundational generative AI training. This voluntary effort simplifies the process for publishers and website operators, who previously had to identify individual crawlers and maintain separate rules for each provider. By adopting a shared signal, operators can communicate the same preference to multiple participating providers at once. OpenAI plans to implement this signal in the coming months and will share details about the implementation. The new signal will not replace existing robots.txt controls, which will continue to apply. This initiative draws on the work of the IETF AI Preferences working group, and OpenAI encourages other AI providers to honor the signal, viewing it as a practical step toward a more open web with clearer ways for website owners to communicate their choices.

(Source：OpenAI)