Many WordPress site owners find themselves puzzled when their main domain performs well in search engine rankings, but their subdomain pages remain invisible. This issue is especially frustrating when traffic and visibility hinge on those subdomain pages being discoverable and indexed. One site’s experience highlights how a deep audit of robots meta tags and canonical URLs revealed the causes of indexing failure—and how resolving them brought back search visibility.
Contents of Post
TLDR:
While the main domain of a WordPress site was indexed and ranking well, subdomain pages were not appearing in the Google Index. A comprehensive audit revealed incorrect robots meta tags and misconfigured canonical tags as the culprits. Once corrected, indexing resumed across subdomain content. Properly managing meta data and canonical elements across subdomains is crucial for SEO consistency.
Understanding the Problem: Subdomain Pages Not Indexed
In this particular case, the website consisted of a main domain (e.g., example.com) and a subdomain (e.g., blog.example.com) that housed blog content, tutorials, and resource pages. Although the root domain experienced solid organic traffic, none of the blog’s pages were appearing in search results—even after weeks of publication.
Initially, the team believed the delay was due to algorithmic crawling frequency. However, further observation using Google Search Console revealed “Discovered – currently not indexed” status for most of the subdomain pages. Something was clearly wrong beyond slow crawling.
The Role of Robots Meta Tags
One of the first things the SEO team checked was the robots meta tag settings on the subdomain. In WordPress, some SEO plugins auto-apply settings globally, which may inadvertently result in <meta name="robots" content="noindex"> being added to non-home pages.
Upon inspection, they found that:
- The homepage of blog.example.com had a proper
index, followdirective. - Internal post pages, however, had
noindex, follow, instructing search engines *not* to index them, but still follow their links.
This mistake was traced back to a theme template that included a hardcoded robots meta tag, combined with the SEO plugin’s default settings taking precedence inconsistently across content types.
Solution: They removed the hardcoded meta tag and allowed the SEO plugin (in this case, Rank Math) to manage robots meta directives. Additionally, a rule was set to ensure all published posts carried the index, follow tag unless explicitly customized per post.
Canonical Tags: The Hidden Villain
After adjusting the meta tags and revalidating affected pages in Search Console, some pages started indexing, but others still weren’t. The next item under scrutiny was the canonical tag, a meta element meant to suggest to Google the preferred version of a URL.
The audit uncovered a surprising pattern: multiple blog pages had canonical tags pointing back to the main domain’s homepage!
<link rel="canonical" href="https://example.com/" />
This canonical tag told Google that the content on blog.example.com/some-article was just a copy of example.com. As a result, the subdomain content was not considered unique or index-worthy.
This misconfiguration happened due to:
- The use of a shared header template between the main domain and subdomain.
- Plugin default settings not properly detecting the current host domain of the page.
Solution: Developers modified the canonical tag logic to dynamically reflect the current page’s full URL, including subdomains. For WordPress users, this can be managed using filters and hooks provided by SEO plugins.
The Impact of a Proper Audit
After correcting both the robots meta and canonical tag issues, the team resubmitted affected URLs to Google via Search Console using the “Inspect URL” tool. Within 7-10 days, a majority of the previously missing blog posts began to appear in the index.
In terms of Search Engine Optimization (SEO), the blog regained visibility, and subdomain traffic began to match projections based on content value. Organic impressions increased by over 250% in the following month.
Key Lessons Learned
For SEO professionals and developers managing complex WordPress structures, especially those involving subdomains, the following strategies are crucial:
- Audit Meta Tags Regularly: Use tools like Screaming Frog or Ahrefs to crawl the site and validate robots meta directives.
- Proper Canonical Tag Configuration: Ensure every page has a canonical URL pointing to itself unless rel=canonical consolidation is intentional.
- Limit Hardcoding: Avoid hardcoded robot directives or canonical tags in theme files. Delegate this work to SEO plugins when possible.
- Monitor Search Console: Treat “Discovered – not indexed” as a red flag. Dive deeper when you notice subdomains aren’t behaving like the main domain.
- Test in Staging: Always test theme and plugin modifications related to SEO meta tags in a staging environment first.
Conclusion
WordPress makes web publishing easy, but managing SEO on a multi-layered domain architecture requires diligence. Subdomain pages being excluded from indexing while the main domain ranks well is often not a penalty but a sign of misconfigured SEO elements. This case illustrates the importance of strategic audits, especially of meta robots and canonical tags, to restore and sustain visibility in search results. Anyone operating multiple subdomains should make such audits part of their routine SEO maintenance checklist.
FAQ
- Q: Why was the main domain indexed but the subdomain wasn’t?
- A: The main domain had the correct meta settings, while the subdomain pages had
noindexdirectives and incorrect canonical tags pointing to the main domain, which prevented their indexing. - Q: What is a canonical tag and why does it matter?
- A: A canonical tag tells search engines which URL is the preferred version of a page. If it incorrectly points to another domain or page, it may prevent the current page from being indexed.
- Q: Can misconfigured plugins cause SEO issues?
- A: Yes. Plugins that manage SEO metadata can unintentionally apply global settings across all domains or subdomains, leading to indexing problems if not configured properly.
- Q: How can I check if my subdomain pages are indexed?
- A: Use Google Search Console and enter the individual page URL into the “Inspect URL” tool to see its indexing status.
- Q: Are subdomains treated differently by Google?
- A: Google generally treats subdomains as separate properties, so SEO settings need to be managed independently from the root domain.