The Crawl Journal

Solving Google Indexing Problems with Noindex, Robots.txt, and Canonical Tags

When a page you care about does not show up in Google search results, the first reaction is often frustration. But the cause is usually something you can fix. Three technical settings control what Googlebot sees and how it treats your content: the noindex directive, the robots.txt file, and the canonical tag. Each one works differently, and mixing them up can create the very Google indexing problems you are trying to solve. What the noindex Directive Does The noindex direc