Cloudflare added a setting to block AI training

Cloudflare introduced a new feature called Disallow AI Training, which allows website owners to prohibit the use of their content for training neural networks. The setting is available in the control panel and allows for differentiated access for search robots and crawlers that collect data for generative models.

The technical implementation is based on standards already supported by Google, Microsoft, and Apple. When the option is activated, the system automatically updates the access rules, blocking bots that collect data for training, while maintaining page indexing for search engines. This solves the problem of mixed traffic, where the same crawler performs both search and data collection functions for training datasets.

Resource owners can manage access through a single switch in the security settings. The company emphasizes that this solution is aimed at ensuring transparency and control over intellectual property in the face of growing numbers of autonomous agents and LLM. The tool is already available to all platform users without the need for manual editing of robots.txt files.

The implementation of the feature was a response to the increasing cases of unauthorized use of content for training proprietary models. According to community discussions, this approach allows website owners to maintain visibility in search results without sacrificing data privacy. This simplifies access management for administrators who previously had to block entire IP address ranges or complex User-Agent chains.