Site map for Blogger and how important is it?

I am writing this post in the hope that anyone who will read it will not suffer the same thing that I did.

Fist a little background
I went to webmaster tools (my first time) and was going through all the tools and when I got to Blocked URLs, I noticed I have a huge number under Blocked URLs. This got me a bit curious so I check my robots.txt to see if somehow there are listed URLs there but there are none. Well there should be none because I have not configured it to not crawl any of my posts/pages.

My investigation
I don't understand why I should have a big number of blocked URLs, actually by my estimate, it was my entire blog content. This should not be so because doing a search for site:http://cyberliving.blogspot.com to check if my site is being crawled and indexed shows all my pages. This only means that I have not been blocked or penalized by Google.

I as able to breathe better knowing I am not being punished by Google. But something is really wrong because webmaster tools is reporting some blocked URLs. I checked my index status and imagine my shock to find out that it is showing 0 (zero) total indexed, Blocked by robots show the same number as the Blocked URLs but Ever crawled is showing a steady increase.

I have checked my other blogs if it is showing the same problem (I thought that maybe there was an error on this tool) but my other blogs are showing a good amount of indexed page and none on the Blocked URLs.

The realization of the problem
Well now I am starting to understand things. Making some comparisons on what I did to my other sites compared to my current problem, I have come to realize that I did not make a site map.

I then went ahead and updated to update my site map and voila! all my URLs have been submitted and after a day, I can see that all my URLs have been indexed.

Before this, I have not really considered a site map to be important but after seeing what it has done for this site, I urge everyone that it is one of the first a webmaster needs to do in order for search engines to crawl your site extensively and get indexed properly.

Site Map
I tell you now my fellow blogspot users, the only site map we can use for blogger are RSS feeds. If you go search the internet on how to create and submit a site map for your blogspot site, you will only be disappointed and maybe confused. There is NO WAY you can use an XML or any other file as site map. I am stressing this point to save you the same trouble that I went through.

So RSS feed it is. There is also one good thing about using feeds as site map and that is you do not need to update your site map every time a new page is added to your site which is particularly useful when you are blogging because all you need to worry about is creating your posts and your feeds will do the job of updating your site map.

Fortunately, an RSS feed is already made for you when you create your own blog and it is powered by Atom. Its location can be found in your robots.txt file and your robots.txt file can be accessed through webmaster tools -> Health -> Blocked URLs (or you may type in the address bar http://YOURSITENAME.blogspot.com/robots.txt). In there you can already see the content of your robots.txt file. Look somthing like this:

Sitemap: http://YOURSITENAME.com/feeds/posts/default?orderby=UPDATED

Submit your Site Map
Copy everything after the .com (blue fonts from above) then go to Webmaster Tools -> Optimization -> Site Maps -> Click ADD/TEST SITEMAP. Paste on the input box the content of your clipboard (feed location that you copied from robots.txt). You are done. After submitting a sitemap the list of submitted maps will be updated together with its status.

If you still have some doubts or question, you may post them on the comments section and so long I will try to helps as much as I can.

Back to home page

Other Posts
Unlimited Surfing, What is the catch?
Making a link on Blogger's Dynamic View landing page