Hay it is a bot party on my web site

I am seeing a lot of traffic from the search engine bots the last few days, the usual suspects include googlebot msnbot, and a new one is gigabot, and from what I can find from a google search on gigabot, is that it is a new search engine that wants to have more of an index then google (good luck to them.) The gigabot one is smashing my site, it has itself downloaded almost a gigabyte of stuff in the last 2 alone, all the pictures and videos I have put up are also being read by that bot, why a bot cares about pictures and videos I do not know, but it is reading them also.

 Post details 

Categories: Web Site
Tags: No Tags
Published on: June 14, 2005

 Comments (2) 

  1. Tim says:

    I think it’s a spammer, I’ve banned gigabot from my site. Most of the hits were coming from Hurricane Electric which I know isn’t interested in being a search engine.

  2. Tom Marshall says:

    Gigabot has been overloading my hosting providers server all week – just from trawling one site, and has crashed it several times already.

    It’s gotten far more aggressive recently, and seems to ignore robots.txt too.


 © 2024 - Michael P. O'Connor