How To Get Google To Index Your Site With The Coverage Report - Semalt Knows The Answer
It is time to take a deep dive into your Search Console Index Coverage report to understand how we can get Google to crawl and index your site faster. At Semalt, we have several professional technical SEO staff, and they are all conversant with using the Google Search Console Index Coverage report.
If you have a technical SEO "Expert" who doesn't use or understand this tool, get a new one. The GSCIC report provided an in-depth understanding of:
- Which URLs on your website have been crawled and indexed by Google, and which URLs are yet to be crawled.
- It also explains why the search engine has chosen which URL it crawls or not.
The report appears to be relatively simple as it uses traffic signals color scheme to represent its results.
- Red light (Error): This shows that the page has not been indexed.
- Yellow (Valid with a warning): this indicates that there might be some issues that need fixing. If you have time, you can fix them up. However, they aren't critical, and the page may be indexed.
- Green (Valid): this says all is good, and your page has been indexed.
One other result is the big gray Zone, which has been excluded.
As we read further, we realize that the road rule seems to be written in a Googlish language. However, we could translate the status types in the indexing and drive up our organic performance.
SEO impacting issues in the index coverage report
The key here is to ensure that you do not only focus on the errors. More often than not, the significant SEO wins will be buried in the gray area mentioned above. Here are some index coverage report issues that genuinely matter for SEO. These items have been listed in priority order, so you know what and where needs your attention most.
Discovered content is not currently indexed
This happens because the URL is known to Google by links or an XML sitemap, and it is in the crawl queue. The issue here is that Googlebot is yet to crawl the URL. This indicates that there's a crawl budget issue.
How can we fix this? If there are only a few pages that fall into this category, we can trigger a crawl manually by submitting the URL(s) in Google Search Console. If there are a significant number of URLs, we will invest more time into a long term fix of your website's architecture. This will include the site taxonomy, URL structure, and internal link structure. Doing this will solve your crawl budget problems from their sources.
Crawled - currently not indexed
Sometimes, Googlebot will crawl a URL and find that its content is not worthy of being included in its index. This is common due to quality-related problems like having outdated content, thin or irrelevant content, doorway leading pages, or user-generated spam. If your content is deemed worthy, but it isn't indexed, the chances are that the problem is as a result of rendering.
How can we fix this? A quick solution will be to review the contents of your pages. When you understand what Googlebot thinks, your page's content is now valuable enough to be indexed. Then you figure out whether or not the page needs to exist on your website.
Suppose the webpage isn't useful to your website, 301 0r 410, the URL. If it is important, modify the content on the page and add a non-index tag until you can solve the issue. If you have a URL that's based on a parameter model, you can stop the page from getting crawled by using some practice parameter handling techniques.
To avoid such delays, you can use server-side rendering. This allows all essential components of content to be presented in the initial HTML. This should include critical elements of your SEO, such as page headings, structured data, your main content and links, headings, and canonicals.
Duplicate content without user selected canonical
This happens when Google considers the page to be duplicate content, but it isn't marked with a clear canonical. Here, Google has decided that this page shouldn't be canonical, and because of that, it has been excluded from the index.
To fix this, you will need to explicitly mark the correct canonicals. Ensure you use the correct rel=canonical tags for every crawlable URL on your website. This enables you to understand which pages are selected as the canonical by Google, we will need to inspect the URL in Google's Search Console.
Duplicated, submitted URL, which isn't selected as canonical
This is caused by a similar situation listed above. The only difference here is that you specifically asked for the URL to be indexed.
To fix this, you will have to mark the correct canonical using a rel=canonical link. This should be used on every crawlable URL on your website. You should also ensure you only include canonical pages in your XML sitemap.
Google chooses a different canonical
In this case, you've placed your rel=canonical links, but Google doesn't find this suggestion and appropriate, so it chooses to index a different URL as the canonical.
To fix this, you will need to inspect the URL to see the canonical URL Google has selected. If you feel Google has made the right choice, change the rel=canonical link. If not, you would have to work on the website architecture and reduce the amount of duplicate content. You should also send stronger ranking signals to the page you wish to be canonical.
Submitted URL not found (404)
The request made for a page doesn't exist. To fix this, you will need to create the URL or completely remove it from your XML sitemap. This problem is easily avoidable by following our guide on the XML sitemap.
Here, Google bots have taken issues with the redirect. This is mostly caused by having a redirect chain of five or more URLs long, redirect loops excessively long URL, or an empty URL.
We can fix this by using debugging tools such as the lighthouse. A status code tool such as httpstatus.io can also be used to understand what is stopping the redirect from performing as expected and showing how the problems identified can be solved.
It is important that you ensure that your 301 redirects are always pointing directly to the final destination. If you need to edit the old redirects, it's better to edit them.
Server error (5xx)
This occurs when the server returns a 500 HTTP response code or an internal server error code when they are unable to load individual pages. This can be caused by a wide variety of server issues, but more often than not, it is caused by a short server disconnection that stops Google bots from crawling the URL.
How you approach, this is partly dependent on how often it occurs. If this happens once in a very long while, there's nothing to worry about. After some time, the error will go away. If the page is important to you, you can recall Googlebot to the page after the error by requesting an index on the URL.
If the error is reoccurring, you should speak to your engineer, teach the team and hosting company to improve their services. If the problem persists, consider changing your hosting company.
Overall, we believe in preventing a problem rather than finding solutions for it. With our well thought out website architecture and robot handling, we often produce absolutely clean and clear Google Search Console index coverage reports. However, we sometimes take on clients who had their site built by others, so we can't develop the site from scratch. For this reason, we check on this report regularly and see to what extent Google has crawled and indexed the site, after which we take notes on the progress.
At Semalt, we have a team of experts who are here to serve you. Do you have any problems related to any of the items listed above? Or do you have any questions relating to SEO and site indexing? We are more than happy to help you iron out the details. Our services also extend to maintaining your site, which involves fixing these issues.