What Is Indexing?

How Indexing Works

Indexing is the second of Google’s three search stages, following crawling and preceding the serving of results. Once Googlebot has fetched and rendered a page, the indexing systems take over. In Google’s words, “Google analyzes the text, images, and video files on the page, and stores the information in the Google index, which is a large database.” The index is what Google searches when a user types a query — not the live web. A page that is not in the index cannot appear in results, full stop.

Indexing is not passive storage. During this stage Google performs canonicalization: it identifies pages with duplicate or near-duplicate content and consolidates them. As the documentation puts it, “we first group together the pages that we found on the internet that have similar content, and then we select the one that’s most representative of the group.” That representative page is the canonical, and it is the version Google stores and shows. If you have several URLs serving similar content, Google will pick one — and it may not be the one you would have chosen.

The most important thing Google says about this stage is a warning: “Indexing isn’t guaranteed; not every page that Google processes will be indexed.” Crawling makes a page eligible for indexing, but the indexing systems still evaluate it and can decline. This is why Search Console distinguishes between “Discovered — currently not indexed” (Google knows the URL but hasn’t crawled it) and “Crawled — currently not indexed” (Google fetched it and chose not to store it). The two point to very different problems.

Indexing is also where Google extracts and understands the page’s content — parsing its text, alt attributes, and key tags, and assessing what the page is about. That understanding, stored alongside the page in the index, is what the serving stage later matches against queries. A page can be indexed yet still rank poorly because Google understood it as being about something other than the queries you targeted, which is a content and relevance issue rather than a technical one. Indexation is the floor for visibility, not a guarantee of it.

What Determines Whether a Page Gets Indexed

Indexation hinges on a handful of technical and content factors:

• Indexability directives — A noindex meta tag or header keeps a page out of the index even if it’s crawled perfectly.
• Canonical signals — Conflicting or incorrect canonical tags can cause Google to index a different URL than you intend, or consolidate away a page you wanted kept.
• Duplication — Pages that collapse into an existing content group may be dropped in favor of the representative version.
• Content value — Thin, boilerplate, or low-value pages are the most common victims of “Crawled — currently not indexed.”
• HTTP status — Only pages returning a success status are indexed; a 4xx or 5xx response keeps the page out or removes it.

Example of Indexing

Google’s How Search Works guide is the primary source that documents indexing in the search team’s own language, and two of its statements define the entire discipline. First, on what the stage produces: Google “stores the information in the Google index, which is a large database,” making explicit that search runs against this stored index, not the live web. Second, on the limits: “Indexing isn’t guaranteed; not every page that Google processes will be indexed.” Those two sentences together explain why a technically healthy, crawlable page can still be invisible — processing is not the same as inclusion.

The canonical-selection mechanic is equally concrete. Google describes grouping “pages that we found on the internet that have similar content” and then selecting “the one that’s most representative of the group.” A worked consequence: suppose an e-commerce site serves the same product at /shoes/model-x, /sale/model-x, and /model-x?ref=email. Google may group all three, judge them near-duplicates, and index a single representative URL. If your internal links and canonical tags don’t consistently point at the version you want ranked, Google’s choice of representative can differ from yours — and the other two URLs effectively disappear from the index by design, not by error.

Google’s HTTP status codes documentation adds the removal side of the mechanic: it states that “Google doesn’t index URLs that return a 4xx status code, and URLs that are already indexed and return a 4xx status code are removed from the index.” So indexation is not a one-time achievement. A page earns its index slot by being crawlable, canonical, distinct, and returning a healthy status — and it keeps that slot only as long as those conditions hold. The example matters because it comes straight from Google stating the rule, not from inference about ranking behavior.

Indexing vs Crawling

The distinction is practical, not academic. If a page was never crawled, you have a discovery problem — fix links, sitemaps, and robots.txt. If it was crawled but not indexed, you have a quality or canonical problem — Google saw the page and passed on it. Treating an indexing verdict as a crawling delay is the most common way teams waste weeks chasing the wrong fix.

Frequently Asked Questions

The Bottom Line

Indexing is Google deciding your page is worth remembering. Crawling gets the page read; indexing gets it stored and made eligible to rank. Because Google grants it selectively and never guarantees it, earning indexation means giving Google a page that is distinct, canonical, and clearly worth a place in the database.