Managing your WordPress search engine visibility setting is a fundamental step in controlling whether Google indexes your website content. WordPress includes a built-in option to discourage search engines from indexing your site, but understanding what this control actually changes in your markup and how Google interprets those instructions prevents unexpected visibility issues.
How the Search Engine Visibility Setting Works
In WordPress, the Reading Settings documentation explains that checking the Discourage search engines from indexing this site box asks search engines not to index the site. When this option is checked in modern WordPress installations (since Version 5.3), WordPress generates a specific meta tag into the head section of your site source if wp_head is used: <meta name=’robots’ content=’noindex,nofollow’ />. This tag instructs search engine spiders to ignore your site.
Enabling this checkbox also modifies ping behavior and publishing notifications. It stops pings to ping-o-matic and any other RPC ping services specified in your Update Services, and it hides the Update Services option entirely on the Writing Settings screen. Crucially, the setting still allows normal visitors to browse your website. The setting does not block access to your site; it is ultimately up to search engines to honor your request.
How Google Processes the Noindex Rule
According to Google Search Central documentation on blocking indexing, noindex is a rule set with either a meta tag or an HTTP response header used to prevent indexing by search engines that support the rule, such as Google. Google supports combining the noindex rule with other directives, such as joining a nofollow hint with a noindex rule in a robots meta tag.
When Googlebot crawls a page and extracts that tag or header, Google drops that page entirely from Google Search results, regardless of whether other sites link to it. However, Google notes an important operational limitation: Google web crawlers have to crawl your page in order to see meta tags and HTTP headers. If a robots.txt file blocks the URL from Google web crawlers, crawlers cannot see the meta tag. Google explicitly states that specifying the noindex rule directly in a robots.txt file is not supported by Google. Therefore, to unblock your page from Google so crawlers can detect tag changes, you must edit your robots.txt file if it is preventing crawling.
Workflow to Manage and Verify Search Engine Visibility
To inspect, update, or remove the search-engine visibility setting on your site, follow this concrete workflow. For this workflow, consider a hypothetical local business website that has completed development and is ready for public search indexing:
- Review the visibility checkbox: Navigate to your WordPress Reading Settings. When your site is ready for search engines, ensure the Discourage search engines from indexing this site box is unchecked, then save your changes. This removes the automated noindex,nofollow meta tag from your pages.
- Inspect page source: As an optional practical suggestion, view the HTML source code of a published page to confirm that the noindex meta tag is no longer present within the head section.
- Verify crawler access: Check that your robots.txt file does not block Google web crawlers from reaching your URLs, because Googlebot must be able to crawl the page to identify your updated indexing instructions.
- Test with Google tools: As an optional practical suggestion, use Google’s URL Inspection tool to test whether your noindex implementation is correct and see the HTML that Googlebot received while crawling the page.
- Request recrawling: You can request that Google recrawl a page using the URL Inspection tool. Note that depending on the importance of the page on the internet, it may take months for Googlebot to revisit a page naturally if recrawling is not requested.
- Monitor ongoing status: You can also use the Page Indexing report in Search Console to monitor the pages on your site from which Googlebot extracted a noindex rule over time.
Technical Limitations to Keep in Mind
While the WordPress visibility checkbox provides a straightforward toggle, site owners should keep documented limitations in mind. The WordPress setting relies on the wp_head hook being present in your active theme to generate the meta tag. Furthermore, Google warns that some search engines might interpret the noindex rule differently, meaning a page could still appear in results from other search engines. Using the checkbox asks search engines not to index the site, but it does not provide access restrictions for ordinary human visitors.