So I have a content scrapper that visits every evening to scrap my "new" listings page and generates 100's of false click throughs while he's at it. The ip tracks to India, it changes every night but the first 6 are always the same .. 117.204.x.x
I've been pulling the "new" page down in the evenings, hoping they'll get the hint and shove off. I thought about blocking the 117.204 range, but wouldn't that be a bit too wide of a block? Got any creative suggestions to annoy them away without losing other visitors?
Is there a specific user agent their scraper always uses? .htaccess user agent blocks can be constructed.
Do you have IP geolocation installed? If so, is it always the same city? If it's not a city you get many legit people from, blocking the city is an option at Admin -> Members -> Settings.
0/5
1
2
3
4
5
Sorry, you don't have permission to post posts. Log in, or register if you haven't yet.
Comments on Blocking Content Scrappers
Member
Usergroup: Customer
Joined: Nov 12, 2004
Total Topics: 9
Total Comments: 20
Posted Sep 24, 2012 - 10:30 PM:
So I have a content scrapper that visits every evening to scrap my "new" listings page and generates 100's of false click throughs while he's at it. The ip tracks to India, it changes every night but the first 6 are always the same .. 117.204.x.x
I've been pulling the "new" page down in the evenings, hoping they'll get the hint and shove off. I thought about blocking the 117.204 range, but wouldn't that be a bit too wide of a block? Got any creative suggestions to annoy them away without losing other visitors?
developer
Usergroup: Administrator
Joined: Dec 20, 2001
Location: Diamond Springs, California
Total Topics: 61
Total Comments: 7868
Is there a specific user agent their scraper always uses? .htaccess user agent blocks can be constructed.
Do you have IP geolocation installed? If so, is it always the same city? If it's not a city you get many legit people from, blocking the city is an option at Admin -> Members -> Settings.