Google SERP Scraping: How to Validate Suspect Rankings
Google SERP scraping is a useful monitoring signal, but agencies should validate abrupt rank changes with matched request context, raw URL evidence, browser checks, and Search Console trends before reporting a loss.
· 13 min read
Nozzle’s October 21, 2026 update examined 29.9 million Google SERPs and reported that 22.6% of its judged page-one-or-two result sets met its poisoned-SERP criteria after September 13. For SEOs and agencies, the payoff of validating Google SERP scraping is avoiding a client-facing ranking emergency when the underlying observation may instead be a collection, parsing, redirect, or request-context problem.
A changed result page does not establish that Google deliberately shows fake rankings to every bot. It can reflect localization, device differences, session state, automated-traffic controls, incomplete rendering, /goto URL handling, parser breakage, transient experiments, or a genuine ranking change. The practical job is to preserve the evidence and state the confidence level before recommending expensive work.
What the poisoned-SERP reports actually establish
Nozzle uses the term “poisoned” for SERPs that appear normal but contain an unexpectedly altered set of organic sites compared with the historical result set for that query. Its Poisoned Google SERPs research gives the underlying vendor context for the headline figures rather than treating them as a measure of every Google search.
In its October 21, 2026 update, Nozzle reported that 22.6% of judged result sets had a poisoned page one or page two after September 13, versus 0.28% before that date. The daily rate reportedly peaked at 43.5% on September 30, fell to roughly 4% to 6% on October 14 and 15, and rose again to 35.7% on October 21.
The examples are striking but should not be generalized beyond their samples. Nozzle showed a US desktop query, white label local seo, where 9 of 9 page-one sites changed within three days, and a Queens mobile example where 8 of 10 page-one sites disappeared. Those observations support a data-quality concern; they do not reveal Google’s internal reason for every unusual response.
What the denominator means—and what remains unknown
The 22.6% figure is not “22.6% of Google searches” or “22.6% of all rank-tracker checks.” It is Nozzle’s rate among the SERPs it collected and judged under its own methodology, counting a result set when either page one or page two met its criteria. That denominator matters because page depth, keyword mix, country, device, collection timing, and sampling rules affect any percentage.
The public report describes the comparison as an abnormal change in the expected set of sites, but it should be consulted for Nozzle’s current operational rules, including how it samples keywords, compares historical SERPs, and handles repeated domains. A reader should not assume that a “site changed” means a domain was deduplicated, that subdomains were combined, or that every URL variant was counted separately unless the vendor states that rule.
Nozzle also reported that 67.9% of flagged SERPs returned to their usual sites within three days. That reversal rate is useful for triage, but it is not proof that every reversal was caused by bot handling. Google experiments, data-center routing, local intent shifts, and collection failures can also be temporary.
Why GrowByData and Semrush do not necessarily conflict
A second dataset helps test whether one provider’s result is isolated, but it does not make different studies directly interchangeable. GrowByData’s Poisoned SERPs study reviewed approximately 2.1 million SERPs across six US cities, the UK, Ireland, Canada, and Australia from June 1 through October 27, 2026.
GrowByData reported an abnormal-result baseline near 1% from June through August. Its highest weekly rate was 16.0% in Ireland in the week beginning September 28; it reported 3.4% of monitored US SERPs flagged from October 16 to 18 and a latest US weekly rate of 3.5%.
Those percentages should not be compared as though they measure the same population. GrowByData flags pages against historical expectations in its monitored sample, while Nozzle reports its own judged page-one-or-two population. Their query portfolios, cities, devices, rerun schedules, historical windows, domain-deduplication rules, and treatment of SERP features may differ. Neither public report proves deliberate result substitution by Google.
Search Engine Watch’s dated report, “Semrush says Google is feeding rank trackers bad results, but impact is limited”, adds another useful qualification. It reported that Semrush had seen many questionable results beyond page one and characterized the tracked-ranking impact as limited. That can coexist with Nozzle’s finding that page two was affected 1.5 times as often as page one: different providers can observe different mixtures of queries and collection contexts.
Google SERP scraping changes with request context
Google Search is not a static database export. A search for heating and cooling services queens on an iPhone in Queens can differ from the same text requested from a desktop browser on a US data-center connection, even when both tests select the United States.
At least seven variables should be recorded before comparing two result sets:
- Query and modifiers: quotation marks, spelling, operators, and punctuation can change intent.
- Location: country alone is insufficient for local queries; city or regional signals matter.
- Language and interface settings: language can alter both result selection and displayed features.
- Device: mobile and desktop can have different layouts, result counts, local packs, and ordering.
- Session and consent state: cookies, login, prior activity, and consent choices may change what loads.
- Network route: VPNs, shared networks, residential connections, and data-center IPs are not equivalent contexts.
- Time: news, shopping, local, video, and AI features can change rapidly.
A result difference under unmatched conditions is therefore not evidence of rank loss. It is simply a difference. A residential proxy may reduce the gap between a data-center request and a consumer connection, but it does not reproduce an individual user’s history, precise location, browser capabilities, or interactions.
What Google documents about automated traffic
Google’s direct unusual traffic help page names robots, computer programs, automated services, search scrapers, and software that checks how a page ranks as forms of automated traffic. That documented statement is enough to make traffic classification a relevant measurement variable for rank tracking.
It does not say that Google returns intentionally false organic rankings to all automated requests. It also does not establish that any particular anomalous SERP was generated because of bot detection. A CAPTCHA, unusual-traffic message, error response, or empty page is clearer evidence of access controls than a normal-looking page with odd organic results.
Claims such as “Google is altering results for bots,” “the page partially rendered,” or “a bot control produced a plausible-looking poisoned SERP” should be presented as hypotheses unless the collector has direct evidence. For example, saved HTML, network logs, screenshots, browser-console records, and a reproduced contrast between matched routes can narrow the explanation, but they still may not disclose Google’s internal decision-making.
The safest operational rule is not to increase retry volume or proxy rotation after a disputed result. Preserve the first response, reduce the test to a small query set, and conduct controlled comparisons later. A repeatable anomaly deserves more attention, but disappearance on rerun is only a diagnostic clue—not proof that the original result was invalid—because transient experiments and collection failures can disappear too.
Why /goto URLs can create false ranking losses
On August 26, 2026, Search Engine Land reported Google’s confirmation that it was deploying google.com/goto?url=... redirects as an anti-abuse measure. This article relies on that dated report for the rollout statement; it should not be represented as a Google documentation page unless Google publishes a direct technical specification.
For searchers, the additional redirect hop may be largely invisible. For a collector, it can cause an extraction failure if legacy code expects the raw href to contain a publisher URL. The result can be a false “removed URL,” an apparent duplicate, or a destination mismatch.
A robust collection record separates three values:
- Displayed URL or breadcrumb: the destination shown to the searcher.
- Google click URL: the raw link captured from the rendered result page.
- Resolved canonical destination: the final publisher URL after following the redirect and normalizing parameters.
It should also retain title, snippet, position, feature type, timestamp, response status, and the extraction method. A raw /goto URL is evidence of a redirect format; it is not evidence that a publisher lost a ranking. See Google.com/goto URLs: How to Audit SERP Data Without False Ranking Losses for a focused audit sequence.
A worked example: diagnosing one disputed keyword
Consider an agency report showing a plumbing client falling from position 4 to position 47 for emergency plumber queens at 09:15 America/New_York on November 3, 2026. The captured row contains a Google-owned /goto link, while the client’s Search Console clicks and impressions for the query group have been stable over the preceding 14 days.
That is not enough evidence to report a rank-47 loss. The analyst should first inspect whether position 47 is a resolved publisher URL, an unnormalized redirect, an organic result, an ad, a local-pack component, or a parser-selected element from another feature.
Controlled checks for the disputed row
The analyst records the precise query, New York location setting, English language, mobile device profile, collection timestamp, raw HTML, rendered screenshot, raw click URL, and final destination. They then run three checks over 24 to 72 hours using the same documented context, plus one independent configured source.
Suppose the result is position 4 in two mobile browser checks, appears as position 5 in a second tracking source, and the original tool’s later export resolves the client URL correctly. The appropriate conclusion is: unverified collection or URL-normalization anomaly on the initial run, not confirmed ranking loss.
Suppose instead that all three matched browser checks exclude the client, a second source shows a comparable decline, and Search Console impressions fall across related Queens emergency-plumbing queries for three weeks. That still does not identify a single cause, but the evidence supports escalating to an indexing, competition, local-intent, and page-quality investigation.
A minimum evidence protocol for agency reporting
Screenshots are useful, but a screenshot alone cannot show the request conditions that created it. For every disputed keyword, retain a compact evidence packet rather than a single rank number.
- Record the exact query, timestamp, time zone, country, city, language, device, browser state, and network context.
- Save the rendered result view and, where lawful and operationally appropriate, the response or extracted-result record.
- Label each listing as organic, ad, local, shopping, video, AI surface, or another SERP feature before comparing positions.
- Preserve displayed URLs, raw click URLs, and resolved canonical destinations separately.
- Repeat a limited sample across 24 to 72 hours and compare with a configured independent reference.
- Review Google Search Console impressions, clicks, CTR, queries, pages, and indexation signals over a meaningful period.
Search Console is not a live replica of a SERP and cannot reveal every competitor position. It is nevertheless stronger evidence of a site’s own sustained Google performance than one third-party snapshot. When impressions rise but clicks do not, query mix and SERP features may matter as much as rank; this practical CTR audit explains how to investigate that pattern.
AI visibility requires a separate evidence trail
Google organic positions, AI Overviews, AI Mode, ChatGPT, and Perplexity are separate surfaces. A top-ten organic ranking is not an AI Overview citation, and a brand mention in an answer is not necessarily a linked or attributable citation.
For each answer-engine observation, record the exact prompt, engine, date, location, language, visible answer, cited domains or URLs where shown, and whether the brand was mentioned, cited, both, or neither. Availability and citations can vary by query, geography, account state, and date, so one observation should not become a permanent visibility claim.
Audra’s local-first desktop audit is designed to check page-level AI answer-engine visibility alongside technical SEO, performance, accessibility, best-practice, and link findings. Operationally, that means teams can pair an AI-visibility observation with the audited page and its on-site evidence rather than treating an answer-engine mention as a standalone score. The report should still distinguish what Audra observed from what was manually reproduced and what remains unverified.
This distinction is also covered in AEO vs SEO: A Practical Guide to AI and Google Visibility: the disciplines overlap in useful content and technical accessibility, but their measurements are not interchangeable.
Classify the finding before assigning a cause
A concise classification prevents teams from turning ambiguous data into a confident narrative. The following labels are more useful than calling every anomaly “fake results.”
| Evidence observed | Appropriate classification | Next action |
|---|---|---|
| CAPTCHA, unusual-traffic message, empty page, or repeated errors | Blocked or rate-limited request | Stop retries and retest later in a clean, documented context |
| Different results by city, language, or device | Context variation | Match location, language, and device before comparison |
| Google-owned raw URL with a normal final destination | /goto handling issue | Resolve and normalize the destination URL |
| Abrupt site-set change that later reverts | Unverified transient anomaly | Preserve evidence and repeat limited checks |
| One tracker falls while Search Console and browser checks are stable | Tool-specific measurement concern | Review parser, route, and vendor evidence |
| Sustained Search Console decline plus repeatable browser loss | Probable real visibility change | Audit indexing, intent, competitors, and affected pages |
The key distinction is evidential, not rhetorical. “Confirmed” should require corroboration; “probable” should identify what remains uncertain; and “unverified anomaly” should remain an acceptable client-report outcome when the data does not support a stronger conclusion.
FAQ
Can Google show different results to scrapers than normal users?
Google documents that search scrapers and rank-checking software are forms of automated traffic in its unusual-traffic help guidance. Results can also vary by location, device, language, session, and time. Vendor reports document abnormal-looking result sets, but they do not prove that Google intentionally provides false rankings to every automated request.
What does a poisoned Google SERP mean?
“Poisoned SERP” is Nozzle’s label for a normal-looking results page whose organic site set changes unexpectedly against its historical expectations. It is a vendor measurement classification, not a confirmed explanation of Google’s internal behavior. Review the provider’s sampling, baseline, page-depth, and deduplication rules before comparing its percentage with another vendor’s rate.
Why do scraped Google results contain google.com/goto URLs?
Google’s /goto format adds a Google redirect between a result click and the publisher destination. A collector that stores the raw click URL without resolving it can mistake the redirect for a changed or missing result. Store the displayed URL, raw Google URL, and normalized final destination as separate fields.
How should an agency verify a sudden ranking drop?
Use the identical query, city, language, device, and browser context; save rendered and extraction evidence; and repeat a small sample over 24 to 72 hours. Compare the result with Search Console and, where possible, a second configured source. A vanished anomaly can be informative, but it is not conclusive proof that the first observation was wrong.
Can Audra prove that an AI answer engine will cite a page?
No single audit can guarantee a future citation because answer-engine outputs vary by prompt, date, location, availability, and model behavior. Audra can provide page-level AI-visibility checks alongside site-health findings, while teams should preserve the prompt, observed answer, citations, and verification status when making client-facing AI visibility claims.
Sources
- https://nozzle.io/research/poisoned-google-serps/
- https://growbydata.com/poisoned-serps/
- https://www.searchenginewatch.com/poisoned-google-serps-rank-trackers/
- https://searchengineland.com/google-confirms-deploying-goto-url-redirects-to-search-results-links-485926
- https://support.google.com/websearch/answer/86640?hl=en