Voor het geval dat iemand zich nog afvroeg waarom ik alle MSN Live.com bots op mijn website een IP ban heb gegeven: Yell if Microsoft’s Live.com Spammed You Too.

 

The summary: IP addresses belonging to Microsoft are requesting pages from eKstreme.com and blogSci.com (my science blog) with HTTP referer headers suggesting that the hits were from live.com searches. These referrer headers are spoofed as the keywords from these supposed searches are sometimes in no way related to the requested page.

Met als gevolg dat je inkomsten van Google AdSense dalen.

Annoying as all that is, however, it is still nothing compared to the real clincher here. This bot sent by Microsoft to attack our innocent websites isn’t just downloading the HTML pages… it’s also downloading CSS files and Javascript, including downloading AdSense code from Google. I included a snippet of my logs from Aug 17th to show what I mean here. You can see where the bot, which is spoofing IE7, and never once downloaded robots.txt, hit the page, then downloaded my local .js file, followed by the external css, followed immediately by a visit from the “Mediapartners-Google” bot… which means that Google thought that this rogue bot from Microsoft was actually a visitor to my site.

On AdWords, Google will slow delivery, stop displaying altogether, and raise the minimum bid for those advertisers whose ads have a poor click-through. This means that it makes perfect sense for Google to reserve ads with the better chances of making them money for sites on the content network that have a better chance of the ads actually getting clicked. If this is in fact the case, then Microsoft, in addition to annoying the hell out of me by screwing up my traffic reporting, is also fucking with mine and other people’s income. This is just not acceptable.

En dit is pas het zoveelste geval van MSN Live bots die zich misdragen. Nog steeds volgen ze de regels in robots.txt met willekeur en downloaden ze pagina’s die ze niet mogen downloaden. Toen Live.com net begon, negeerde de MSN bot alle robots.txt instructies, met alle security issues die daarbij horen. Toen ben ik ook begonnen met het blocken van alle MSN bots. Vandaag heb ik er dus weer een IP range aan toegevoegd.

This entry was posted on Sunday, 18-11-2007 at 15:59 and is filed under MSN, Websites Algemeen. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.