Thread: Block The Baidu Spider From My Site?
View Single Post
Old 08-06-2009, 23:26   #2
Acathla
Inactive
 
Join Date: Jun 2003
Posts: 1,354
Acathla has reached the bronze age
Acathla has reached the bronze ageAcathla has reached the bronze ageAcathla has reached the bronze ageAcathla has reached the bronze ageAcathla has reached the bronze ageAcathla has reached the bronze ageAcathla has reached the bronze ageAcathla has reached the bronze ageAcathla has reached the bronze ageAcathla has reached the bronze ageAcathla has reached the bronze ageAcathla has reached the bronze ageAcathla has reached the bronze ageAcathla has reached the bronze ageAcathla has reached the bronze ageAcathla has reached the bronze ageAcathla has reached the bronze age
Re: Block The Baidu Spider From My Site?
You can do this but it will still make server requests - just not index anything:

Code:
SetEnvIfNoCase User-Agent "^Baidu" bad_bot
<Directory />
Order Allow,Deny
Allow from all
Deny from env=bad_bot
</Directory>
You could probably also see about doing a mod_rewrite like:

Code:
RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} ^baiduspider [NC]
RewriteRule .* - [F]
Acathla is offline   Reply With Quote