Tells AI crawlers (such as OpenAI ChatGPT) not to use your website as training data for their Artificial Intelligence (AI) products. It does this by updating your site’s robots.txt to block common AI crawlers and scrapers. This should prevent your content from being used to traing Large Language Models (LLMs).
It blocks these AI crawlers and bots:
- ChatGPT and GPTBot – Crawlers and web browser used by OpenAI
- Google Extended – Crawler used for Google’s Gemini (formerly Google Bard) AI training
- FacebookBot – Crawler used for Facebook’s AI training
- Meta – Blocks crawlers used by Meta AI training
- CommonCrawl – Crawler that compiles datasets used to train AI models
- Anthropic AI / Claude – Crawler used by Anthropic
- Omgili – Crawler used by Omgili for AI training
- Bytespider – Crawler used by TikTok for AI training
- PerplexityBot – Used by Perplexity for its AI products
- Applebot – Used by Apple to train its AI products
- Cohere – Crawler used by Cohere AI training
- DiffBot – Crawler used by Diffbot for AI training
- Imagesift – Crawler used by used by Imagesift for images
- … and more!
Experimental Meta Tags
The plugin adds the “noai, noimageai” directive to your site’s meta tags. These tags tell AI bots not to use your content as part of their data sets. These are experimental and they have not been standardized.
Disclaimer
Note: While the plugin adds these markers, it is up to the crawlers themeselves to honor these requests.