Description

Should publishers block AI bots like ChatGPT, Gemini and Perplexity from crawling their sites? If so, what is the smartest way to do it?

In this session, Eric Shanfelt shows you how generative AI is impacting publication search traffic, which types of publications are most at risk, and a practical framework for deciding whether blocking AI crawlers makes sense for your business. He then breaks down step-by-step how to actually block these bots at the robots.txt and firewall level without accidentally cutting off Google and other legitimate search engines.

You'll learn:

• How quickly generative AI usage is growing and what that means for publisher traffic.
• Which publishers face the highest AI threat level (national news, hobby/enthusiast, B2B, city and regional lifestyle, city and regional business) and why.
• Real examples where blocking AI bots did not impact on organic search or overall visits.
• A simple recipe for blocking AI crawlers in robots.txt and keeping that list updated.
• Why you also need firewall or hosting level rules, not just robots.txt.
• When it makes sense to geo-block your site from countries with heavy bot activity.
• Common mistakes to avoid, including accidentally blocking normal search engine bots.

Whether you run a national news or B2B trade site, or a local consumer or business publication, this episode will help you decide where you stand on AI crawlers and give you a clear, practical playbook for protecting your content.

Learn more at https://nearviewmedia.com/