Till a year ago, the only way you could get your site indexed on Google is by waiting for Googlebot (the software program or spider that Google uses to crawl the Web) to visit your site.
While Googlebot does a fairly comprehensive job of covering the Web, webmasters needed more control over how their site was indexed by the search engine.
In 2005, Google introduced Sitemaps.
Here’s what Google has to say about its Sitemaps: “Search engines such as Google discover information about your site by employing software known as “spiders” to crawl the web. Once the spiders find a site, they follow links within the site to gather information about all the pages. The spiders periodically revisit sites to find new or changed content.
“Google Sitemaps is an experiment in web crawling. By using Sitemaps to inform and direct our crawlers, we hope to expand our coverage of the web and speed up the discovery and addition of pages to our index.”
So what exactly are Sitemaps?
Simply put, sitemaps are files (created in XML) that inform Google about the content or pages on your site. You can create sitemaps by using the Google Sitemap generator or a variety of third-party tools.
What makes sitemaps particularly useful for Webmasters, as well as Google, is the fact that it allows you tell the search engine a lot more than just the URLs of pages on your site.
This includes:
· How often you update the page
· How important a page is in relation to other pages on your site
· When it was last updated
Such information can help both the site as well as Google.
For instance, you may want the Googlebot to revisit your frequently update pages more often than a page that is never updated.
It also gives websites more control over what they feel is important on their site rather than submit to the whim of a software program. For instance, you can tell Google through the sitemaps that your product pages are more important than your Contact page.
A particularly useful feature of Google Sitemaps is the ability to tell Google about pages that would never have been reached by the Googlebot. Most search engine spiders have trouble indexing dynamically generated pages or pages that sit behind a decision point — for instance, pages that can be reached only through a search.
This is a great boon for e-commerce sites because it allows them to submit dynamically generated pages that were till now beyond the spider’s ken.
Sitemaps also gives webmasters a lot of flexibility. If, for instance, your site structure has changed, all you need to do is make those changes in your Sitemaps file, and Google will know about it.
There’s more to Sitemaps
In 2006, Google made some great additions to the Sitemaps program. It now offers webmasters a host of valuable information about how Google views your site.
Getting started is easy as pie. Just submit your URL and verify your site ownership and Google starts to display statistics that can help you tweak your site and make it more Google-friendly.
These include:
Crawl errors: This lists areas of your site that Google had trouble crawling.
Search query information: Google provides information on top queries that returned results from your site and the top queries that directed traffic to your site. You can also get this information for specific regions. For instance, if you only want information about search queries from google.co.in (India).
Page analysis: Google provides information on how it views your site and the words other sites use to link to yours.
Indexing information: Google tells you if your site is indexed or if it had trouble crawling your home page. It will also show you pages from your site in its index, and when the Googlebot last visited your site
Violations: Google will also inform you if it found any violations of its Webmaster guidelines on your site.
Webmasters can use this information to create much more search-engine friendly sites.
For instance, if one of your key phrases figures in top search queries but not in search query clicks then perhaps you need to look at some on page factors (such as title tag to attract more clicks) or focus on improving the ranking for those phrases.
Page analysis, on the other hand, provides a list of words that Google associates with your site and another list of words that other sites use to link to you. This can be a useful tool in determining whether Google views your site through the same keywords that you do. If it does not, maybe you need spend more time on optimizing your site for them.
In conclusion, Google Sitemaps is a very useful tool for Webmasters to tell Google how to view their site and to understand how Google actually views it.
