I got up at around 10 AM today .Nothing unusual as its my routine on holidays. I decided to skip breakfast as i was toooo tired (by sleeping?? ) to walk to the hotel.Ate some fruits , and came online. Being online till 3 am last night , i was sure that there will be nothing new for me there.But there was something….Something to keep me busy for the whole day.There was a mail from my webhoster saying that my bandwidth allocation for this month was almost over…Over?? Its 7th April and with 7 days i used up my 500MB Bandwidth limit??..Confused … Last month it was 200~250 mb for a whole 30days..and all of a sudden within 7 days ,…500MB! I contacted arun(from Sprite host) and told him about that.He too was a bit surprised to hear that.He checked with server stats and confirmed that its true. So how could a site with a few images and a blog use up 500mb in 7 days..Server stats didn’t recognised the hostname/IP from where that suspicious traffic came from, but it did log the usage as “Anonymous users”.So we assumed it as bot activity.He found 10360 requests for .jpg files from my site. ( i have some 10~20 jpg files in my site) So thats where my mbs are going.Bots!#$$$#[email protected]!.. I have a robot.txt banning access to the bots from critical data.But i forgot to add my images folder to that list..
Ok..now blocked images folder also..But it will work only with branded bots..Like google or yahoo…If you are also being hit by many other ‘bots with no name or unfamiliar names, then you may need to use a stronger method, such as access blocks by IP address or user-agent.But my server didnt logged any ips for bots..So i have to tweak .htaccess file now.Not confident enough to do that..But somehow i have to …Else..Soon my blog will display a page “Bandwidth over limit .Site Blocked”
PS: I upgraded my plan from 500MB/month to 1GB / month….