Smetimes you have things you don’t want indexed by Google. What are those things, and how to you avoid them being indexed? The top offender when it comes to stuff being indexed that shouldn’t is duplicate content. Search engines can get confused and dismissive when presented with a lot of highly similar content. When they [...]
When You Don’t Want to be Indexed
June 21st, 2009Don’t Tick Off the Googlebot
March 2nd, 2009Whenever I visualise the Googlebot, I get this weird cross hatched version of a mutant R2D2 combined with Shelob the Spider from the Lord of the Rings. You want your pages to be crawled, but there are a few things you should remember to do – or not to do, as the case may be. [...]
A Word On Sitemaps
June 29th, 2008Googlebot will crawl your pages, as long as they have a link that is known to Google. Then Google will update their index, rank you, and include you at the appropriate spot. Sometimes they will crawl you fairly often; sometimes it takes quite a while. If you have spent considerable time and resources updating and [...]