Cloudflare's September 15 AI bot defaults will decide whether your products appear in AI search: Do you know the setting exists?

On September 15, 2026, Cloudflare will automatically block AI training and Agent crawlers on pages displaying advertising for new domains and free-tier accounts. If your eCommerce store runs on Cloudflare and displays ads or promotional content on product pages, audit your AI crawler settings before the deadline to ensure your products remain visible in AI-powered search and shopping recommendations.

On September 15, 2026, Cloudflare will flip a switch that could make thousands of eCommerce stores invisible to AI-powered search engines overnight. And the merchants affected probably won't know it happened for weeks.

Starting on that date, Cloudflare's default settings for new domains and domains on the free tier will automatically block AI Training and Agent crawlers on any page that displays advertising. For publishers, this is a content-protection win. For eCommerce merchants, it's a product visibility crisis hiding inside an infrastructure setting most have never seen.

The scale of what's changing

To understand why this matters, consider what AI crawler traffic looks like for eCommerce right now.

According to Cloudflare's own May 2026 data, AI crawlers now represent 20.3% of all verified bot traffic. HUMAN Security's 2026 Retail & E-Commerce Bot Benchmarks paint an even more striking picture: retail and eCommerce sites received nearly two-thirds of all AI crawler requests in 2025, with 77% of AI agent and browser traffic targeting eCommerce specifically. These bots crawl retail sites 198 times more per visit than Googlebot.

This traffic isn't just noise. According to JetOctopus, 75% of AI bot activity is driven by user searches, real consumers using AI tools to discover and compare products. When ChatGPT, Google AI Mode, or Perplexity recommend a product, they're drawing on data their crawlers collected from your product pages.

Block those crawlers, and your products disappear from the recommendations.

Why eCommerce stores are uniquely vulnerable

Cloudflare's new defaults are designed with publishers in mind, sites whose primary value is textual content that AI companies have been scraping for training data without compensation. The "ad-supported pages" trigger makes sense in that context: if a page earns revenue from display ads, the site owner probably hasn't consented to their content being used to train AI models.

But eCommerce product pages exist in a grey zone. Many Shopify, WooCommerce, and Magento stores use Cloudflare for CDN, DDoS protection, and performance optimisation. Some of those stores display retargeting ads, affiliate banners, or promotional widgets on product pages. Under Cloudflare's new taxonomy, those pages could trigger the default block . The critical distinction that most merchants miss: Cloudflare now classifies AI bots into three categories, Search, Training, and Agent. The September 15 defaults block Training and Agent bots on ad-supported pages while allowing Search bots through. But the classification of "mixed-use" crawlers, bots that serve both search and training purposes, is where merchants lose control. If a crawler is classified as mixed-use and the page triggers the ad-supported filter, the default is to block.

The critical distinction that most merchants miss: Cloudflare now classifies AI bots into three categories, Search, Training, and Agent. The September 15 defaults block Training and Agent bots on ad-supported pages while allowing Search bots through. But the classification of "mixed-use" crawlers, bots that serve both search and training purposes, is where merchants lose control. If a crawler is classified as mixed-use and the page triggers the ad-supported filter, the default is to block.

For eCommerce stores, blocking an AI agent means blocking the AI shopping assistants that are increasingly placing orders on behalf of consumers. Blocking a mixed-use crawler means potentially disappearing from AI-generated product recommendations across multiple platforms simultaneously.

The visibility decision you didn't know you were making

Here's what makes this particularly dangerous for eCommerce teams: this is an infrastructure decision with commercial consequences, but it lives in a DNS dashboard rather than a marketing tool.

Most eCommerce managers, even technically capable ones, manage their store through their platform admin panel. Cloudflare settings are handled by a developer, an agency, or were configured once during site setup and never revisited. The AI crawler controls were added to Cloudflare's dashboard in July 2026, and unless someone actively checked, the settings are at their defaults.

For existing Cloudflare customers, the current defaults are more permissive. But here's the trap: if an eCommerce store changes its Cloudflare plan, adds a new domain, or if Cloudflare extends the new defaults to existing domains in a future update, the store's AI visibility could change without any action from the merchant.

There's no alert in Shopify, WooCommerce, or Adobe Commerce that tells you "AI crawlers are being blocked at the CDN layer." The store continues to function perfectly. Traditional SEO tools continue to report normal Google rankings. But AI search engines gradually stop recommending your products because their crawlers can't access your pages.

What merchants should audit before September 15

The audit is straightforward, but it requires cross-functional awareness:

1. Check your Cloudflare AI crawl settings. Navigate to your Cloudflare dashboard and review the AI bot classification settings. Ensure that Search bots are allowed on all product pages. Decide intentionally whether Training and Agent bots should access your product catalogue.

2. Verify your robots.txt configuration. If your store uses Cloudflare's managed robots.txt feature, confirm it isn't blocking AI user agents like GPTBot, ClaudeBot, or OAI-SearchBot. If you're on Shopify, check whether Cloudflare's settings conflict with Shopify's own Web Bot Auth signed crawler requests.

3. Identify ad-supported pages. Audit which product pages display advertising elements that could trigger the default block. Retargeting pixels, affiliate banners, and promotional widgets may all qualify.

4. Monitor AI referral traffic baselines. Before September 15, establish a baseline of traffic from AI-powered search engines. Without a baseline, you won't be able to detect a decline.

5. Test crawler access from the outside. Verify that AI crawlers can actually render your product pages by checking the cached versions in AI search tools.

Why continuous monitoring is the missing layer

The fundamental problem with the Cloudflare default change isn't the policy itself; it's that merchants have no visibility into whether AI crawlers are successfully accessing their product pages after any configuration change.

This is where AuditIQ eCommerce monitoring provides the capability most eCommerce teams lack. AuditIQ continuously monitors your storefront from the outside, the same perspective AI crawlers have, and detects when pages become inaccessible, when structured data breaks, or when crawler access changes. When Cloudflare's defaults shift on September 15, AuditIQ can verify whether your products remain visible to the AI search engines that are driving an increasingly significant share of product discovery.

The September 15 deadline is five weeks away. The audit takes an afternoon. The cost of not doing it is invisible: products that quietly stop appearing in the AI-powered recommendations where 75% of bot activity originates from real user searches.

Your products might be perfectly optimised for AI search. This is the core of Generative Engine Optimisation (GEO), ensuring your products remain discoverable and recommendable inside AI answers and shopping agents. But if the infrastructure between the AI crawler and your product page blocks the connection, none of that optimisation matters.

Don't wait to discover the gap after your AI referral traffic drops. [Book a free AuditIQ demo] and see how continuous storefront monitoring keeps your products visible to AI crawlers.

FAQs

1. Does this affect my store if I'm on a paid Cloudflare plan?

The September 15 defaults apply to new domains and free-tier accounts. Existing paid customers are currently on more permissive defaults. However, if you add a new domain, change your plan, or if Cloudflare extends the new defaults to existing accounts in a future update, your settings could change without warning. Auditing your current AI crawler configuration is worth doing regardless of your plan.

2. How do I know if my product pages count as "ad-supported" under Cloudflare's new taxonomy?

Cloudflare's trigger includes retargeting pixels, affiliate banners, and promotional widgets, not just traditional display ads. If your product pages load any third-party advertising technology, including common retargeting tags from Google, Meta, or affiliate networks, they may qualify. Check your Cloudflare dashboard's AI bot settings and review which pages display any form of advertising element.

3. If AI crawlers are blocked, will I still rank in Google Search?

Yes, Cloudflare's new defaults allow Search bots through while blocking Training and Agent bots. Your Google rankings are unaffected. What changes is your visibility in AI-generated product recommendations, such as those from ChatGPT, Google AI Mode, and Perplexity, which rely on Agent and mixed-use crawlers that may be blocked under the new defaults.