The MSN Bot (formerly of MSN Search, now Bing) is causing problems for website owners by making it look like they have some connection with porn sites.
It’s an ongoing problem; Microsoft’s search engine has been generating fake information that makes site owners think they’re getting porn traffic for over two years now.
Microsoft’s software that visits web pages to index them for the search engine is a crawler known as MSN Bot. When it gathers a web page, it will on occasion leave behind “referrer” information in a web site’s logs
Referrer information is a way for the web browser to tell a web site the last page it viewed before coming to the site.In other words, it’s supposed to show from what page the crawler followed a link to get to yours. A lot of this referrer data has turned out to be fake – which relieves some as they now know they are not being linked to from porn sites, but incenses others as the inference remains that they could be associated with such sites.
Microsoft’s crawler is unusual in that crawlers don’t normally report referrer data. MSNbot isn’t doing searches on its own Bing search engine, then following listings to visit new pages, but that is what the data left on indexed sites makes it look like!
Microsoft admitted they had a fake referrer problem in December 2007, but said it was fixed. Apparently not – many site owners are still complaining about the appearance of being linked to from porn sites and even for ranking for porn terms! )They aren’t actually, but the information MSNbot left behind when visiting says they do.)
Microsoft needs to fix this problem before people lose faith in Bing.
