What Is Noindex Tag?
Written byFlavio Amiel Founder, Roborank
A noindex tag is a directive that tells search engines not to show a page in their results. It is delivered either as a robots meta tag in the page’s HTML head — <meta name="robots" content="noindex"> — or as an X-Robots-Tag: noindex HTTP response header. When Google crawls the page and reads the directive, it drops the page from its index.
- The two valid forms are the HTML meta tag
<meta name="robots" content="noindex">and the HTTP headerX-Robots-Tag: noindex; the header can noindex non-HTML files like PDFs. - Noindex only works if the crawler can reach the page. Google states that if a page is disallowed in robots.txt, indexing directives “will not be found and will therefore be ignored.”
- Noindex controls indexing; nofollow controls link following.
content="noindex"still lets Google follow the page’s links;content="none"equalsnoindex, nofollow. - Because Google must re-crawl a page to see a newly added noindex, deindexing is not instant — the page stays in results until the next crawl reads the directive.
How the Noindex Tag Works
Search engines index by default: crawl a page, and unless told otherwise, it becomes eligible to appear in results. A noindex directive overrides that default. When Googlebot fetches a page and finds a noindex rule — in the HTML or in the HTTP response — it removes that URL from the index at the next processing pass and keeps it out for as long as the rule is present.
There are two equivalent delivery mechanisms, and Google treats them the same. The first is the robots meta tag, placed in the <head>:
<meta name="robots" content="noindex">
You can target a specific crawler by swapping the name value, for example name="googlebot". The second mechanism is the X-Robots-Tag HTTP response header:
X-Robots-Tag: noindex
The header is the only option for non-HTML resources — a PDF, an image, a spreadsheet — because those files have no <head> to hold a meta tag. It is also convenient when you want to apply the rule across many URLs at the server level.
The one hard prerequisite is crawlability. Google is explicit: “If a page is disallowed from crawling through the robots.txt file, then any information about indexing or serving rules will not be found and will therefore be ignored.” A noindex the crawler cannot reach is a noindex that does nothing. This is why noindex and a robots.txt disallow are alternatives, not partners — pick one based on whether your goal is to stop crawling or to stop indexing.
Directive Values
The robots meta tag and X-Robots-Tag accept a shared vocabulary of directives, and they can be combined with commas:
noindex— do not show this page in search results.nofollow— do not follow the links on this page.none— shorthand equivalent tonoindex, nofollow.noarchive— do not show a cached link for the page.nosnippet— do not show a text snippet or video preview in results.unavailable_after: [date/time]— do not show the page in results after the specified date.
When two rules conflict, Google applies the more restrictive one. Note that noindex and nofollow are orthogonal: content="noindex" on its own still permits Google to follow the page’s links, which is often exactly what you want for a thin page whose outbound links still matter.
Example of the Noindex Tag
The most instructive worked example is the failure mode Google documents directly. In Block search indexing with noindex, Google spells out the exact trap: “you must not block the page from crawling. Otherwise, the crawler won’t see the noindex rule, and the page can still appear in search results, for example if other pages link to it.”
Walk through what happens. Suppose you have a staging URL you want gone, so you add <meta name="robots" content="noindex"> to the page and add Disallow: /staging/ to robots.txt on the same day. Googlebot, obeying the disallow, stops fetching /staging/. It therefore never re-reads the HTML, never sees the noindex, and — if any external page links to the staging URL — can keep the URL in its index as a bare link with no snippet, exactly the outcome you were trying to prevent. The two signals cancel each other.
The correct sequence follows straight from the mechanism. Add the noindex and leave the page crawlable. Wait for Google to re-crawl and drop the page — you can confirm removal with the URL Inspection tool in Search Console. Only after the page has left the index, if you also want to conserve crawl budget, is it safe to add the robots.txt disallow. For urgent cases on your own verified property, Google’s Removals tool can hide the URL within about a day, but that hold lasts only around six months, so it buys time for the durable noindex to take effect rather than replacing it.
The general principle: a noindex tag is a message to the crawler, and a message only lands if the crawler can read it. Every real-world noindex problem is, at bottom, a page the crawler could not reach.
The single most common way I see noindex fail is people blocking the page in robots.txt at the same time they add the tag — usually with the honest logic of "I want this gone, so I’ll hide it every way I can." It backfires completely. Google can only obey a noindex it has actually read, and a robots.txt disallow stops it from ever fetching the page to read it. So the page stays indexed, often showing up as a bare URL with no description, and nothing you do to the tag matters because Googlebot never comes back to see it. If you want a page out of the index, do the opposite of your instinct: leave it crawlable, let the noindex be seen, and only after it has dropped from results should you consider blocking the crawl. Order of operations is the whole game here.
Noindex Tag vs Nofollow
| Noindex Tag | Nofollow | |
|---|---|---|
| Controls | Whether the page appears in the index | Whether the page’s links are followed |
| Scope | The page itself | The links on (or from) the page |
| Syntax | <meta name="robots" content="noindex"> |
<meta name="robots" content="nofollow"> or rel="nofollow" on a link |
| Typical use | Thank-you pages, internal search results, staging URLs | Untrusted or paid links, comment sections |
| Effect on the other | Alone, still lets Google follow links | Alone, still lets the page be indexed |
The two are frequently confused because they live in the same content attribute, but they answer different questions: noindex is about this page’s presence in results, nofollow is about the links leaving this page. Combine them as content="noindex, nofollow" (or none) only when you genuinely want both.
Frequently Asked Questions
What is the difference between noindex and nofollow?
Why is my noindex page still showing in Google?
Should I use a meta tag or the X-Robots-Tag header for noindex?
Does noindex remove a page from Google immediately?
The Bottom Line
A noindex tag is the correct, reliable way to keep a page you control out of search results: state the directive in the HTML head or the HTTP header, and make sure the page stays crawlable so the engine can actually read it. It is not the same as nofollow, which governs links rather than the page, and it is not instant — the page leaves the index only once the directive is next crawled.
Sources
- Robots meta tag, data-nosnippet, and X-Robots-Tag specifications — Google Search Central
- Block search indexing with noindex — Google Search Central
Roborank checks whether your noindex directives are actually being seen — catching pages that are noindexed but robots.txt-blocked, or indexed when you meant to hide them.
Check your indexing rules →Rank & Cash — the weekly SEO breakdown
One practical teardown a week on ranking in search and getting cited by AI. No fluff.