Re: [tor-talk] Ahmia search engine works normally again

[scfith riseup <scfith@xxxxxxxxxx>]

First, not sure why you want to list .onion domains. The key here is that they are HIDDEN services. But I am sure you have reasons.

Second, if you do list .onion domains, know that they will be collected.

Third, provide a simple path of machine-readable content to download so that bots won’t kill your server, like a list.json file that is up to date so your server is not ruined.

my 2 cents.

> On Dec 7, 2016, at 10:40 AM, Nurmi, Juha <juha.nurmi@xxxxxxxx> wrote:
> 
> Hi,
> 
> Ahmia.fi and msydqstlz2kzerdg.onion went offline for a short while. Someone
> flooded the onion site with multiple requests per second: with randomly
> selected user agent strings someone was generating GET
> msydqstlz2kzerdg.onion/address/<somethingherethatischanging> requests.
> 
> Constant flood was similar to this:
> [07/Dec/2016:04:11:04 +0100] "GET /address/... HTTP/1.1" "OS"
> [07/Dec/2016:04:11:04 +0100] "GET /address/... HTTP/1.1" "like"
> [07/Dec/2016:04:11:04 +0100] "GET /address/... HTTP/1.1" "NT"
> 
> As a results, Ahmia tried to load the page /address/ over and over again
> and this exhausted CPU of my front-end server.
> 
> How I fixed this:
> 
> 1) I separated ahmia.fi and msydqstlz2kzerdg.onion to run
> independently with their own processes
> 2) I disabled msydqstlz2kzerdg.onion/address/ completely,
> https://ahmia.fi/address is still available
> 3) I adjusted several performance settings and cache
> 4) I added one more CPU core to the front-end server
> 
> Ahmia search engine works normally again. Still, I hope that this flood
> stops.
> 
> Peace,
> Juha
> -- 
> tor-talk mailing list - tor-talk@xxxxxxxxxxxxxxxxxxxx
> To unsubscribe or change other settings go to
> https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-talk

-- 
tor-talk mailing list - tor-talk@xxxxxxxxxxxxxxxxxxxx
To unsubscribe or change other settings go to
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-talk