2008 October

What is Bad bot?

BAd bot varies depending on who you will ask, they are the spiders that do a lot more harm than good on your website (ie: an email harvester). A website ripper on the other hand are offline browsing programs that a surfer may unleash on your website to crawl and download every one of its pages for offline viewing. In both cases, both your website’s bandwidth and resource usage are jacked up as a result, this can sometimes to the point of crashing your server. Bad bots typically ignore the wishes of your robots.txt file, so you’ll want to ban them using means such as .htaccess.

Bookmark Us

Posted under Linux Hosting

This post was written by admin on October 18, 2008

Tags: bad bot, what is bad bot

Blocking users and websites Via htaccess

You can block users or sites that originate from a particular domain is another useful trick of .htaccess.

For blocking access by referrer in .htaccess requires the help of the Apache module mod_rewrite to make out the referrer first. This module is installed by default on most of the servers . So, to deny access all traffic that originate from a particular domain (referrers) to your website, use the following code given below:

Steps to Block traffic from a single referrer:

RewriteEngine on
# Options +FollowSymlinks
RewriteCond %{HTTP_REFERER} badsite\.com [NC]
RewriteRule .* – [F]

Steps to block traffic from multiple referrers:-

RewriteEngine on
# Options +FollowSymlinks
RewriteCond %{HTTP_REFERER} badsite\.com [NC,OR]
RewriteCond %{HTTP_REFERER} anotherbadsite\.com
RewriteRule .* – [F]

Bookmark Us