What is a Noindex Tag and When to Use It?
A noindex tag tells search engines not to include a page in their index. Here is how it works, when to use it, and the mistakes that cause problems.
16 May 2026 · 6 min read
A noindex tag is an instruction that tells search engines not to include a page in their index. A page with a noindex tag can still be crawled and linked to, but it will not appear in search results.
The most common form is a meta robots tag in the <head> of the page:
<meta name="robots" content="noindex" />
You can also combine it with other directives:
<meta name="robots" content="noindex, nofollow" />
nofollow tells crawlers not to follow the links on the page. noindex on its own still allows link equity to flow through the page's outbound links.
How does noindex differ from robots.txt?
Both noindex and robots.txt control what search engines do with your pages, but they work at different stages and have different effects.
robots.txt prevents crawlers from visiting the page at all. If Googlebot is blocked from a URL in robots.txt, it cannot read the noindex tag on that page either. The URL may still appear in search results (without a snippet) if it receives backlinks.
noindex allows crawlers to visit the page but tells them not to index it. Google reads the tag, confirms the instruction, and drops the page from its index.
The practical consequence: never block a page in robots.txt and noindex it at the same time. If robots.txt blocks the URL, Google cannot read the noindex tag and the page may remain in the index from earlier crawls.
When should you use a noindex tag?
Thin or duplicate content pages
Admin pages, search results pages, filtered views, and tag archives often produce near-duplicate content that adds no value to search results. Noindexing these keeps your crawl budget focused on pages that matter.
Thank you and confirmation pages
Post-purchase confirmation pages, form thank-you pages, and login success screens have no value in search results and can appear strange to users who land on them from Google.
Staging or test pages
If a staging environment or test section of your site is publicly accessible, noindex tags prevent these pages from competing with or diluting your live content.
Private or members-only content
Pages behind a login that are briefly accessible without authentication during a session should be noindexed to prevent accidental indexing.
Parameter-based URLs
URL parameters used for session tracking, ad attribution, or A/B testing create duplicate URLs. Canonicalisation is usually the better fix, but noindex can work for parameter pages that cannot be canonicalised cleanly.
What about the X-Robots-Tag HTTP header?
For non-HTML files (PDFs, images, XML files), the meta robots tag in the <head> does not apply. Instead, you can send a noindex instruction as an HTTP response header:
X-Robots-Tag: noindex
This works for any file type and is the only way to noindex PDFs without editing the document itself.
Common noindex mistakes
Noindexing pages you want ranked
The most damaging mistake is applying noindex to pages that should be indexed. This happens most often during site builds when a staging noindex directive is left in place after launch, or when a CMS noindex setting is applied site-wide by accident.
After any major CMS change or migration, crawl your site and check the index status of your key pages immediately.
Combining noindex with robots.txt disallow
As covered above, blocking a URL in robots.txt prevents Google from reading the noindex tag. Use one or the other, not both. For pages you want removed from the index, noindex is the correct tool. robots.txt disallow is for pages you never want crawled.
Using noindex on a page with important inbound links
If a page has backlinks from external sites and you noindex it, those links still exist but their equity flows to a non-indexed page. The page does not rank, but the links are not entirely wasted. If the page has significant backlink value, consider whether it should be indexed after all, or whether the backlinks should point elsewhere.
Expecting immediate removal
After adding a noindex tag, the page is not removed from Google's index instantly. Google must crawl the page again, read the updated tag, and then process the removal. This typically takes days to weeks depending on crawl frequency.
How to check which pages on your site have noindex
A site crawler gives you a complete view of noindex tags across every URL. Crawly's desktop app crawls your entire site and reports the robots meta tag found on each page, flagging any noindex pages so you can review whether they should be indexed or not.
Noindex tags are precise tools for managing your site's index footprint. Used correctly, they keep low-quality pages out of search results and focus Google's attention on your best content.