Currently, what happens is that sites just ban/blacklist the IPs, often automatically and "forever".
Yep!
Nope. I am well aware of this and it isn't an issue which just popped yesterday or a week ago - it has been going on for months (scraping Google, that is). I am also aware that Scroogle has a limited (I think about 6-7) number of servers.ScroogleScroogle is currently having trouble scraping Google. Maybe Dash Fours problems with it are unrelated to Tor?
What I meant with my initial post though is that Scroogle started blocking tor exit nodes recently - about a week or so ago. I know that, because I tried to access it at the same time (via different machines) and all requests which used Tor exit nodes were timing out (or giving me 502) - without exception, while the "normal" requests (using my own IP address) made at the same time passed through to Scroogle instantaneously! This cannot be a coincidence.
If you do not believe me - see it for yourself - initiate at least a couple of simultaneous requests (so that you can engage as much of Scroogle's servers as possible at one time) using Tor exit nodes and do the same using your own IP address and see what happens.
I will try and prepare such a list of sites I have encountered blocking Tor and post it here.I would like to see a list of sites that block Tor. We can then try to contact them individually to discuss potential alternative strategies.
_______________________________________________ tor-talk mailing list tor-talk@xxxxxxxxxxxxxxxxxxxx https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-talk