They crawl my site just like a robot, but have nothing on their identification, just range of dots.. some said this is an e-mail harvesting robots..
12.44.172.92 - - [14/Nov/2005:03:25:34 +0700] "GET /feeds/categories/5-Daily.rss HTTP/1.0" 200 21758 "-" "Mozilla/4.0 (compatible; MSIE 4.0; Windows NT; ....../1.0 )"
Then I found a solution on this page : How to build a Bot Trap and keep bad bots away from a web site
so first, I install "external PHP application" plugin, include the php file,
and make the page link ( e.g : /trap/index.html )
second, create an invisible link on sidebar with HTML Nugget plugin which
pointing to /trap/index.html
third, add corresponding directory to robots.txt's Disallow option.
( User-Agent: * Disallow : /trap* )
Now the problem is, all mighties ..
I have to include the file blacklist.php.. where should I include it ?
- Tried in index.php, but somehow it fsckup the captcha plugin images..
- tried in index.tpl in default template, I got the "Headers already sent.." stuffs..
It would be nice to have this as additional spam protection..
Just my 2 cents ..