Before you judge your SEO, diff three lists: the pages you intend to rank, the pages Google indexed, and the URLs your sitemap and canonicals actually name
A three-way URL diff shows whether your SEO problem is structural.

Before judging SEO performance, compare three lists of URLs, namely the pages the site intends to rank, the pages Google indexed, and the pages its sitemap and canonicals declare as preferred. This three-way diff separates a content problem from a structural one. Until those gaps are understood, publishing more content risks feeding the same failure.
Why indexing is now selective, not automatic
Google crawls aggressively but indexes selectively, filtering pages for quality and usefulness. A live page is not an indexed page. A URL can be accessible and crawlable, yet still be rejected at the indexing stage. That outcome can reflect a deliberate selection decision.
Using AI to produce content adds to this pressure because Google finds more pages but accepts a smaller share of them. What has Google actually chosen to keep? If indexing must be earned rather than assumed, a site needs a way to audit that choice against what it meant to publish and promote.
Why most SEO judgment is premature without a structural audit first
When traffic flattens, the reflex is to diagnose a content shortage. That diagnosis skips a prerequisite. Additional content helps only if the marginal page gets indexed, gets attributed to a query, and does not cannibalize a page already ranking. On several sites in the multi-site audit described in the brief, all three assumptions failed at the same time. Publishing more in those cases added pages while leaving the reasons existing ones could not perform unfixed.
The site therefore needs to establish what share of the intended content is visible, and where the missing share falls out. A page excluded from the index presents a different problem from an indexed page losing to an unintended variant. Both can look like weak SEO in a traffic chart. They do not call for the same response.
But should better content fix indexing on its own? Quality is necessary, but it is not sufficient. Canonicals, sitemaps, internal linking, and URL variants can suppress good content regardless of its editorial merit. Structural diagnosis must come before content decisions because those signals affect which page gets evaluated and shown. What does that diagnosis look like in practice?
What the three lists are and where each one comes from
The three-way diff begins with independently constructed inputs. No single tool supplies the finished comparison. List 1 contains the URLs the site intends to rank, drawn from the CMS, analytics landing-page reports, Search Console URL exports, and any paid or campaign landing pages. The founder or SEO lead must deliberately decide which belong. This list expresses editorial and business intent.
List 2 shows what Google has actually indexed, using the Pages report in Google Search Console, filtered to the sitemap. It reflects Google's acceptance rather than the site's wishes. That filter also defines the scope of the initial comparison, so indexed URLs outside the sitemap remain relevant when investigating undeclared pages. A query export cannot substitute for this page-level inventory. Its fixed row limit can cover less than half of total impressions on some properties, while page-level data does not have that query-export limitation.
List 3 contains the site's formal declarations, drawn from every XML sitemap file and every canonical tag in page source. The sitemap URLs and canonical targets must remain distinguishable within this inventory so their contradictions stay visible. Even an internally consistent declaration remains a strong suggestion for Google, because canonical tags are advisory.
On 11 September 2026, the Pages report lost several days of data from June 2026 for every property. John Mueller said that Google does not back-fill indexing data. Saving CSV archives on a regular schedule therefore preserves the history of these three inventories, whose differences on a real site provide the diagnostic signal.
How to run the diff and what a healthy ratio looks like
To turn the inventories into a reproducible audit, start in Search Console's Pages report and filter to the sitemap. Divide the number of pages indexed by the number submitted to get an initial index efficiency ratio. The formula is IER = Indexed ÷ Intended Indexable. The submitted count represents the denominator only insofar as the sitemap matches the intended indexable inventory. Otherwise, that disagreement is itself part of the audit.
An IER below roughly 70% is a warning that adding content is no longer the useful first lever. New pages may join the ignored pile. The ratio does not explain the cause, and a higher ratio does not prove that the right pages rank. It provides a starting point for locating the gap.
Next, export the Pages CSV and compare URLs in a spreadsheet, stripping protocol and trailing slash for grouping by canonical path. Preserve the original URLs so the variants remain identifiable. Groups with more than one row expose potential self-competition. Using page-level performance data, sum clicks across each group to quantify the performance distributed among those variants.
Then check the four homepage variants formed by http or https and www or non-www. They should resolve to one canonical host, with alternatives reaching it through one 301. A 307, redirect chain, or two live variants flags a consolidation problem. For the top queries by impression, compare the URL Google serves with the URL intended to rank. A mismatch points toward a miscanonicalized, mislinked, or redundant page, although the query export still describes only its visible slice.
Finally, compare the sitemap and canonical inventory with the indexed inventory. Look for sitemap URLs that redirect, canonicalize elsewhere, or have a noindex tag. These declarations work against each other. The indexed count should roughly match the canonical URL count, and a large gap deserves investigation. The comparison produces four membership gaps, plus any conflicts inside the declarations themselves.
What each class of mismatch actually means
An intended URL missing from the indexed list establishes an absence whose cause needs investigation. The Search Console status supplies the next distinction. “Crawled – currently not indexed” means Google crawled the page but did not retain it, pointing toward a quality or signal problem. “Discovered – currently not indexed” means Google has not prioritized crawling it yet, directing attention toward crawl budget or internal linking. Treating both as requests for better prose confuses different stages of the process.
The reverse gap is an indexed URL the site never intended to rank. This is index bloat, but its consequences depend on what those pages do. If several target the same query cluster, they compete and dilute authority. An unintended, low-quality variant may then outrank the preferred page. More content aimed at that same cluster can deepen the overlap.
A declared URL missing from the index asks another question. Why has Google not honored the site's signal? This calls for page-level investigation through the URL Inspection tool. Because an intended page may also be declared, this category can overlap with the first. The additional evidence is that the site's formal preference exists but has not produced acceptance.
An indexed URL absent from the declarations points in the other direction. Google has accepted something the sitemap and canonicals do not formally claim. Legacy URLs, faceted navigation variants, and orphan pages reachable only through external links can fall into this category. These pages may be diluting authority or creating duplicate-content signals outside the owner's intended inventory. Their presence is not evidence that the planned pages are healthy.
A canonical conflict is a mismatch within List 3. A sitemap names one URL while that page's canonical points elsewhere, giving Google competing signals about the preferred representative. Google may choose a page the site did not intend. Search Console has more than a dozen non-indexed statuses, so even these five diagnostic classes are not interchangeable fixes. The same visible symptom, a page not ranking, can originate in rejection, crawl prioritization, competition, undeclared variants, or contradictory declarations.
Why fixing the structure before touching content also matters for AI answer engines
The same prerequisite extends to AI answers that retrieve from search indexes. A page absent from the specific index an engine relies on cannot be retrieved from that index for citation. The qualification is essential. A page missing from Google's index may still be in a Bing-based engine's sources, so the Google diff cannot show what is visible in every engine.
Indexing is therefore an entry fee. Ranking on Google and receiving a citation in ChatGPT, Perplexity, Gemini, or an AI Overview are different outcomes driven by different signals. Google's AI Mode and Perplexity draw heavily from conventional top-ranking results, while ChatGPT draws a much smaller share from that pool. A Google-only strategy leaves an important blind spot.
To measure the next stage, teams need a fixed cadence of buyer-intent prompts across each engine. Mention rate, citation rate, share of voice, and accuracy should remain separate measures. A mention is not a linked citation. Engines can name brands far more often than they link to them, and that gap varies by engine. Because their citation sources overlap so little, coverage of one platform creates a misleading baseline for the others.
The structural audit belongs before that measurement as part of the same effort. Otherwise, poor citation performance can invite content changes while the relevant retrieval prerequisite remains unresolved. Yet the reverse assumption is also unsafe. An indexed page with no citations may need a closer look.
How to keep the diff current after the first audit
The inventories begin drifting as soon as content is published, URLs change, or Google updates its index. The audit therefore needs a recurring cadence. Archive the Pages report CSV on a fixed schedule, then refresh the comparison against the intended pages and current declarations. Treat a falling IER as a leading indicator that calls for structural investigation before a conclusion about content quality.
A reported decline still needs interpretation before escalation. Is it an index change or a reporting artefact? A real drop is visible in more than one source, with performance-report impressions falling, URL Inspection confirming that pages are excluded, and crawl stats showing less Googlebot activity. A reporting artefact may appear in a single report, affect every property at once, and be flagged by other accounts within hours.
For teams publishing at scale, automation can keep the comparison useful as new pages continually enter the sitemap. A standing watcher can flag newly submitted URLs not yet indexed, canonical conflicts introduced by CMS changes, and sitemap URLs that have acquired noindex tags since the previous pass. These flags identify questions for investigation. A newly submitted page waiting for indexing still needs to be read differently from a page that was indexed before but is now excluded.
The goal is a live comparison available before the next content decision, campaign launch, or migration. At that point, the team can distinguish a missing page from a rejected one, an unintended competitor from a preferred URL, and a coherent declaration from a contradiction. That is the basis for a defensible SEO judgment. Content decisions can then address the gaps that remain after the site’s intent, declarations, and actual indexation have been reconciled.


