December 1, 2006

Web Marketing Newsletter

Search Engine Sitemaps
How to build a site map and why you need MORE than one

Greetings,

My question today is, how's your sitemap?

Every website should have a sitemap, which is a page in your website (or if you have lots of pages in your web your sitemap could be more than 1 page) with a hyperlink to every page in your web. And, every page in your web should have a link to the sitemap. I recommend a static site map, even if you have a dynamic web. You can look at my humble sitemap as an example

www.roncastle.com/sitemap.htm

The purpose of a sitemap is to make it easy for search engine spiders and robots to find all of the content in your website so that you can get your content in the search engine index. Even relatively unimportant pages of content ARE important because page count makes a difference, especially with Google. When you get over 100 pages in the Google index, some really good things start to happen to your ability to gain organic rankings.

There is a new kind of sitemap that is VERY important to your website, a sitemap in XML, which is a very specific form of formatted text similar to html. Early this year, Google started what they call Google Sitemaps BETA, which is a free online service within your free Google account (you can open one if you don't have). The idea Google had was to get you to create and upload an XML sitemap of your website and then register it with them. You tell Google the file name, they go check the format and approve it. In exchange for your time and labor Google will then show you some of the things they know and see when they crawl and index your website, things like 404 file not found errors, for example.

This past month www.Yahoo.com and MSN www.Live.com have come together in conjunction with Google and www.sitemaps.org to agree on and approve a standard protocol for XML sitemaps. This is big news from a search engine optimization perspective and is important to every web owner if you are competing in organic search.

An XML sitemap in your web makes it at least 10 times as easy for search engine spiders and robots to find new or updated content. Rather than crawling every page in your web, all they have to do is crawl the XML file and then go look at the changes or additions. This saves big time on hardware costs (magabucks) for the engines and also makes it faster to keep search results current. In an online world with billions of web pages, this makes a huge difference.

You need an XML sitemap in your web if you want to be a top notch competitor.

How do you make an XML sitemap? There are a number of online sitemap generators, which may work OK for small websites with less than 100 pages of static content. I have tried a number of them. NONE of them have been 100% reliable - they generate sitemaps that meet the protocol but don't necessarily find all the pages in your website. This defeats the purpose.

After 6 months of searching around and testing, I have found the best program available thus far to crawl or spider your website and generate a compliant XML file for Google, Yahoo and Live.com. Last week I used the program to create a sitemap for a dynamic ecommerce website with over 5,900 pages.

If you are interested in creating an XML sitemap for your website, and you absolutely should be, I would be pleased to do this for you or for your webmaster, but it's not a freebie. For a small website, the cost will be $40-$50. For a large dynamic web, it might cost $100-$200 or more depending on the time it takes to crawl the web. If you do not have a static html or asp sitemap, I will also generate a list of URL's that you can paste into a static page in your web to make a static sitemap. I recommend that you have both.

As part of the deal, I will make you or your webmaster totally independent from me so that you can get the program that I use (for free) and use it to keep your XML sitemap up to date. I will double check your work and give you full instructions on how to get your sitemap approved and validated by Google.

Every time I make a change in my website, I create a new XML sitemap, upload it to my website and tell Google it has changed. I have regenerated by sitemap 4 times in the past week. I also do this for all of my web and SEO clients. As a result, I have seen a reduction in time of 2-3 days in getting new content crawled and indexed with Google, from 7-10 days to 5-8 days. If you are doing ecommerce and adding or changing last minute items for holiday shoppers, this is a big deal? I think so.

If you are interested in my offer send me an email with your URL(s) for your website(s) and if you like, I will take a quick look and give you a estimate on cost. You can pay me via PayPal.