What Is a Noindex Tag?
A noindex tag tells search engines to exclude a page from results without blocking crawling. How it differs from robots.txt, with a real example.
A noindex tag is an HTML instruction (via meta robots tag or X-Robots-Tag HTTP header) telling search engines not to include a specific page in their search results, even if the page can be crawled. It’s the correct tool for keeping a page out of search entirely, distinct from robots.txt.
Why noindex tags matter
Noindex tags keep low-value or duplicate pages out of search results without deleting them, protecting a site’s overall quality signals from pages that shouldn’t compete for rankings but still need to exist for functionality, like internal search results, thank-you pages, or filtered category variants. Accidentally leaving a noindex tag on an important page after launch is a frequent and costly technical SEO mistake, common when a staging environment’s noindex setting gets carried over to production and nobody notices until organic traffic mysteriously stalls. Unlike robots.txt, a noindex tag requires the page to actually be crawled for Google to see and respect the instruction, blocking crawling via robots.txt while also expecting a noindex tag to work is a contradiction that prevents the noindex from ever being read.
How noindex tags work
- The tag is added either as
<meta name="robots" content="noindex">in the page’s HTML head, or as an X-Robots-Tag in the HTTP response header for non-HTML files. - The page must be crawlable for the search engine to actually see and read this instruction, it can’t be blocked in robots.txt at the same time.
- Once crawled and the tag is read, the search engine excludes the page from its index and, over time, removes it from any existing search results.
- The page remains fully accessible to human visitors and other crawlers that may not honor the directive, it’s a request to well-behaved search engines specifically.
Common noindex use cases
- Internal search results pages, which offer no unique value to external searchers and can generate infinite low-quality URL variations.
- Thank-you or confirmation pages after a form submission, necessary for the user flow but irrelevant to search.
- Staging or duplicate content not yet ready for public search visibility.
- Thin tag or filter archive pages in e-commerce that generate many low-value URL combinations.
Noindex vs. robots.txt vs. canonical tag
| Method | Prevents crawling? | Prevents indexing? |
|---|---|---|
| Noindex tag | No, page must be crawled to read the tag | Yes |
| Robots.txt | Yes, for compliant bots | Not guaranteed |
| Canonical tag | No | Consolidates signal, doesn’t exclude |
Real example
For example, a site might add a noindex tag to its internal search results pages or a thank-you page after a form submission, since these pages have no value to searchers but still need to exist for site functionality, keeping them out of the index without removing the pages themselves.
Noindex tags and AI search
A noindexed page is generally also excluded from AI Overview citation and most AI crawler training data collection, since these systems largely draw from the same underlying search index or respect the same directive conventions. Some standalone AI crawlers use their own separate rules, so a page meant to be fully excluded from all AI systems may need additional, crawler-specific robots.txt directives alongside the noindex tag.
FAQ
Can I use robots.txt and a noindex tag together on the same page?
Not effectively. If robots.txt blocks crawling, Google never sees the noindex tag in the first place, since it can’t read a page it’s not allowed to crawl. Use one or the other, not both.
How do I check if a page has a noindex tag?
View the page source and search for “robots” in the meta tags, or use Google Search Console’s URL Inspection tool, which explicitly reports the page’s indexing status and directives.
Does a noindex tag remove a page from search results immediately?
No, it takes time. Google needs to re-crawl the page, see the tag, and process the removal, which can take days to weeks depending on the site’s crawl frequency.
Can visitors still access a noindexed page?
Yes. A noindex tag only affects search engine indexing, the page remains fully live and accessible to anyone with the direct URL or an internal link to it.
Related terms
Never combine noindex with a robots.txt block on the same page, that’s a contradiction, not a stronger signal. Pick the right tool for what you’re actually trying to do.