If you use rank tracking tools to monitor your Google visibility, those tools are under growing pressure. Google has stepped up efforts to block automated SERP tracking queries, Search Engine Journal reports, and it argues the driver may go well beyond rank checkers. It points to Cloudflare data published September 30, 2026: daily AI agent requests on Cloudflare’s network grew by more than 1,700% over the past year, and for the first time, more than half of all internet traffic is now non-human.
Why does automated search traffic cost Google more than it used to?
The core issue is cost. AI Mode results require LLM inference (the process of generating an answer by running a large language model across a query) across multiple related queries and reasoning steps, not just a match against pre-indexed pages. Search Engine Journal, reporting October 5, 2026, notes this makes each AI Mode response computationally more expensive to serve than a traditional ranked result.
In Google’s Q2 2026 earnings call published July 22, 2026, CEO Sundar Pichai mentioned cost reduction multiple times: “Thanks to our engineering and hardware optimizations, this quarter we reduced the cost of AI Mode responses to its lowest level since launch, even as we’ve brought more advanced AI capabilities.” Google’s APIs were processing approximately 22 billion tokens per minute at the time of that call, up from 16 billion the prior quarter.
SEJ’s argument: every automated query competes with real users for compute, and because AI-driven results cost more to serve, Google may have a stronger financial incentive to filter automated traffic out.
What may be driving Google’s crackdown on SERP tracking?
The evidence of the crackdown so far comes from the rank tracking side. Ryan Jones, developer of the SERPrecon search data service, wrote that “we’re getting to the point where nobody will have rank tracking anymore because of the crazy amount of AI scraping going on,” and that “Google is blocking all the AI trackers AND AI itself… and the rank trackers are getting caught up,” as quoted by SEJ.
Google’s public actions target rank tracking tools, but according to Search Engine Journal’s October 5, 2026 report, Google has also published research on adversarial content networks where attackers adjust their content based on Google’s ranking signals. Monitoring SERP results is central to that process, and automated querying is the mechanism.
AI agents add a complication that traditional crawlers (automated programs that systematically fetch web pages) do not. Cloudflare’s September 2026 post says AI agent traffic follows human routines, with a weekly rhythm and a dip over the summer holidays. These agents are typically acting on behalf of a real person completing a real task. Cloudflare notes that turning an agent away “may be turning away the person who sent it,” which makes blanket blocking a poor solution for publishers and potentially for Google as well.
What Cloudflare’s traffic data shows
Cloudflare’s blog post from September 30, 2026 provides the most detailed public snapshot of this shift. The figures cover traffic patterns across Cloudflare’s network.
- At the end of 2024, Cloudflare handled 63 million HTTP requests per second on average. By September 2026, that had nearly doubled to 115 million, with peaks above 150 million.
- Daily requests from AI agents grew by more than 1,700% over the past year.
- For the first time, more than half of internet traffic is now non-human.
- Some heavily crawled categories, including Retail, Computer Software, IT & Services, and Financial Services, saw human traffic decline as much as 40% in less than one year.
- The share of crawler requests attributed to AI training rose from 22% in Spring 2025 to 52% by June 2026.
Cloudflare describes this as the breakdown of a 30-year web arrangement: publishers allowed search engines to crawl their content in exchange for human visitors who generated ad revenue or paid for subscriptions. AI answer engines now summarize pages without routing users to the source, costing publishers bandwidth without the traffic that monetizes it. According to Cloudflare, revenue per request is falling while costs are rising.
What this means for AI-search visibility
The practical upshot is straightforward: if Google gets better at blocking automated queries, your ability to measure where your content actually appears in search, including in AI-generated results, weakens at exactly the moment when AI-driven results are reaching more users than ever.
Consider a concrete example. A brand manager today might run automated checks to see whether their site appears in AI Overviews (Google’s AI-generated answer panels in search results) for their key terms. If Google’s anti-automation measures expand to cover those checks, the answer to that question becomes harder to get reliably. Not easier.
The 52% AI training crawler figure from Cloudflare points to a practical trap for publishers. Many sites have configured rules to block crawlers broadly under “AI” labels. But AI training crawlers (which collect content to build models) and search or retrieval crawlers (which help your content appear in AI answers) have different purposes and different implications for visibility. A site that blocked indiscriminately in Spring 2025 may have cut itself off from appearing in AI-driven search results by mid-2026, without realizing it.
More broadly, disruption to SERP tracking accelerates a measurement shift already underway. Traditional rank position becomes a less reliable proxy for visibility when AI-generated answers, not ten blue links, are often what users see first. The question moves from where do I rank for this keyword? to am I cited in AI answers, and how often, and in what context? Those are fundamentally different problems requiring different tools and data sources, and Google’s push against automated querying makes both harder to answer reliably.
In Hingewise’s view, the brands most exposed right now are those still treating rank position as their primary visibility metric while doing little to track presence in AI-generated answers. With AI Mode surpassing 1 billion monthly active users as of Google’s Q2 2026 earnings call, the gap between rank-based measurement and actual user-facing visibility is widening. Rank data is becoming a partial picture of search at the same time it is getting harder to collect, and at the same time it is becoming less representative of how users actually encounter results. That combination is worth taking seriously before measurement gaps become strategic blind spots.
What to check before adjusting your tracking or bot-blocking setup
- Audit which crawlers you are currently blocking: separate AI training crawlers from search crawlers and AI agent traffic. They are not the same category.
- Confirm whether your rank tracking tool has disclosed how it handles Google’s anti-automation measures and whether its data remains reliable under those conditions.
- Check whether your analytics platform can distinguish AI agent traffic from human traffic and from traditional bots.
- Monitor human traffic trends by channel, particularly if you operate in Retail, Software, IT, or Financial Services, where Cloudflare data shows declines of up to 40%.
- Identify which URLs AI agents are fetching most frequently. This can indicate which content is actively being used to answer queries in AI systems.
- Track presence in AI Overviews and AI Mode results separately from traditional organic rankings, as these represent distinct visibility signals with different measurement requirements.
The open question is how Google will calibrate its defenses as AI agents grow into a more commercially significant portion of web traffic. As Cloudflare notes, an agent completing a task on behalf of a real person is effectively a customer. How Google, rank tracking providers, and publishers adapt to that distinction is worth watching closely.
Sources:
Search Engine Journal, “Google Is Fighting SERP Tracking. AI Agents May Make It Worse,” October 5, 2026: searchenginejournal.com
Cloudflare Blog, “The Internet has a second audience,” September 30, 2026: blog.cloudflare.com
Google Blog, “Q2 2026 earnings call: Remarks from our CEO,” July 22, 2026: blog.google
Lam Nguyen · Hingewise
This article was drafted with AI assistance from the sources cited above and checked against them before publishing. The "What this means for AI-search visibility" section is Hingewise's editorial analysis, built on the frameworks and positions we use in our GEO work. How we work: Editorial policy.