Cloudflare, a major internet infrastructure provider powering about 20% of global web traffic, has rolled out a significant policy change: as of July 2025, it is blocking AI web crawlers from scraping content by default unless website owners explicitly permit them. This marks a turning point in the landscape of AI model training and content creator rights.
By default, AI crawlers (bots used by companies such as OpenAI, Google, Anthropic, Meta, and more to collect content for large language models) are now blocked from accessing any website using Cloudflare’s services unless permission is granted by the site owner.
This shift means AI companies can no longer freely scrape huge swathes of the Internet for data without direct consent or compensation for the content creators.
Want to go deeper on AI?
📖 Buy AI Playbook
📩 Get my weekly LinkedIn newsletter, Human in the Loop.
🎓 Level up with the CPD Accredited AI Playbook Diploma
📞 Let's talk about AI training for your team: digitaltraining.ie or publicsectormarketingpros.com if you are in government or publics sector.
