Recently I noticed an odd user-agent in my access logs:
Mozilla/4.0 (compatible; MSIE 5.0; Windows NT; DigExt; DTS Agent
This appears to be an address harvester called Beijing Express E-mail Address Extractor.
I've got 3-5 hits per day from this user-agent, from various IP addresses. The interesting thing is that this bot only ever requests the root document. It doesn't seem to follow any links! How, then, does it find new webpages to strip? What if I had my address buried on an interior page? What if I had it on a style sheet or in a javasript file?
So I'm slightly annoyed at having my pages scanned for addresses. Sure, the bots won't find any, but I object in principle. I've asked my sysadmin to block this user-agent. I don't know if he will, and I'm not even sure if that measure would even be useful. After all, it would only stop one bot.
Marie says:
Hi. I found you in my logs and just wanted to thank you for linking to my page.
Nice blog, and I love your domain name! Very cool. :)
Laurabelle's Blog says:
Bots baby
At the time that I wrote my last entry about banning nasty user-agents, I tried blocking them with mod_rewrite, but