Search Engine Optimization and Digital Media Technologist

How to get Google to crawl your site faster?

Normally there are two possible reasons why Google is spidering your website slow.
The first reason can be that there no or very few quality backlinks to your website found by Google. Google consider your site importance with quality links it finds, the more quality links it finds, the higher points Google associate with your site and more frequently your site will be crawled.
The second reason why Google is slow in spiedering your site might be that your server response is slow. Google point out these issues in Google Webmasters Tools. You may please get understanding from my post “How to use Google Webmaster Tools”. You can see these details in the “Crawl rate” overview of the Diagnostic tab:

Normally there are two possible reasons why Google is spidering your website slow.

The first reason can be that there no or very few quality backlinks to your website found by Google. Google consider your site importance with quality links it finds, the more quality links it finds, the higher points Google associate with your site and more frequently your site will be crawled.

The second reason why Google is slow in spiedering your site might be that your server response is slow. Google point out these issues in Google Webmasters Tools. You may please get understanding from my post “How to use Google Webmaster Tools”. You can see these details in the “Crawl rate” overview of the Diagnostic tab:

Crawl faster

My advice: set it to “Faster”, but don’t stop with that: make your server react faster, either by upgrading your hosting, your hardware, or whatever is needed. Chances are that when Google is suffering long load times, your users are as well.

Now if both of these aren’t right for you, Google has found many great links to your site and it’s not telling you it would like to crawl faster, there is one solution to make sure it crawls your new articles or pages even faster: sitemaps.

I don’t use them on this site myself, as I want to know how Google crawls my site without me interfering in the process. However, if you think you have something to gain by Google crawling your new pages faster, an XML sitemap is a plausible solution.

Do consider though that you’re throwing data away, you’ll never know if and when Google would have found a page by itself if you hadn’t put it in the sitemap.