Re: Google 403 error pages

[Anthony DiPierro <or@xxxxxxxxx>]

On 12/16/05, Izzy <atorima@xxxxxxxxxxx> wrote:
>  I don't think this problem is due just to occasional "spyware, root
> kits, malware", etc. despite the statement on the Google page. My guess
> is that the Google servers are taking an unusual high volume of hits
> from the Tor exit nodes probably because, the "Bad Hats" have been using
> Tor to try to manipulate Google search ranking, ad-sense,  etc. Just a
> guess, but I think a likely scenario. In that case- A technical solution
> for the lack of ethics on the part of some Tor users is what might be
> required. or..eventually..Tor will be blocked, banned or,
> outlawed...which could be the ultimate goal of, some.
>
I figure someone has just been running a bot through Tor, hitting
Google.  Google throttles the number of requests an IP can make over a
certain time period, to stop bots that do this.  Running the bot
through Tor is a good way to get around that throttle, for a while,
but then you wind up screwing everyone else.

>  Meanwhile- The error page has been coming up so frequently these past
> few weeks that I have replaced the default 'Google' search in Firefox
> with the "Scroogle" scraper. They are not blocking the Scroogle proxy...
> yet.
>
> http://www.scroogle.org/
>
> -Izzy

Yeah, I've switched to scroogle too.  I didn't mention it, because I
figured whoever is running that bot might be listening to this mailing
list, and maybe scroogle will block Tor (or Google will block
Scroogle) next.

Anyway, one of the reasons Scroogle isn't usually blocked is because
they've written software to automatically distribute their requests to
come from multiple IPs and to go to multiple different sites on the
Google end.  The different Google sites distributed throughout the
world apparently don't coordinate the throttling mechanism.  All of
this is described on the scroogle site.

If Google decides to get more intelligent with their blocking it
wouldn't be hard to block the scroogle site.  But as long as scroogle
stays relatively low key that probably won't happen.

Anthony