Re: [tor-talk] Load Balancing/High Availability Hidden Services

[s7r <s7r@xxxxxxxxxx>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Hi Donncha,

Your idea is indeed very interesting. I am eager to see it in
practice. If you have the time, me and TheCthulhu (cc'ed) could
provide the infrastructure and necessary system administration hours
to test this.

While your current solution is a way to use load balancing and
redundancy, which means spreading the resource usage over multiple
machines, checkout my questions about maximizing a single HS server as
close as possible to its total capacity:

https://lists.torproject.org/pipermail/tor-talk/2015-March/037173.html

Please email directly if you want to start this, we will document each
step and share the results with everyone here. Maybe this will give
some hints for writing a proposal for load balancing and high capacity
hidden services which could be merged with the proposal of second
generation hidden services.

On 3/12/2015 3:22 PM, Donncha O'Cearbhaill wrote:
> 
> 
> On 11/03/15 17:40, George Kadianakis wrote:
>> MacLemon <tor@xxxxxxxxxxx> writes:
>> 
>>> Hoi!
>>> 
>>> I'm looking into ideas for creating “load balanced” or “high
>>> availability” hidden services. Mostly pertaining to web servers
>>> serving large-ish static files. (Let's say 5-100MB each.)
>>> 
>>> Load balanced as in not all requests end up at the same box to
>>> speed up downloads. High availability as in the service is
>>> still available if one box goes down or is taken offline for
>>> maintenance.
>>> 
>>> So, not exactly your usual distributed-cluster setup.
>>> 
>>> 
>>> From what I understand it would not make sense to run the same
>>> HS Key on multiple boxes since the descriptors would overwrite
>>> each other every few minutes.
>>> 
>>> I don't think one can do something like Round-Robin DNS with
>>> HS.
>>> 
>>> So the only way I can imagine this to work is a central
>>> redirection node that know about all the nodes and more or less
>>> intelligently/randomly 302 redirects each file request to a
>>> known-to-it server.
>>> 
>>> This still leaves a single-point-of-failure in form of the
>>> redirection server but would at least distribute the traffic
>>> load across multiple servers and cope for nodes coming and
>>> going.
>>> 
>>> Has anyone done something like this?
>>> 
>> 
>> An application-layer load balancer like HAProxy might be able to
>> help you.
>> 
>> Unfortunately, there is not something equivalent to DNS Round
>> Robin for hidden services yet. There are some ideas on how to do
>> this on the Introduction Point-layer, but a proposal still needs
>> to be written. For further reading: 
>> https://lists.torproject.org/pipermail/tor-dev/2014-April/006788.html
>>
>> 
https://lists.torproject.org/pipermail/tor-dev/2014-May/006812.html
>> 
> 
> I've been thinking a little about this problem. It seems like one
> simple solution would be to find a way of combining the 
> descriptors/introduction points from multiple Tor HS instances into
> one hidden service descriptor from which the client can pick intro
> points at random.
> 
> To implement such a solution like that, there needs to be a means
> for the hidden service instances to communicate with other. They
> can then either selected a common set of intro points or combine
> the individual sets of intro points selected by each instance.
> 
> In one straight forward implementation a hidden service operator
> could set up a Tor relay and a hidden service on each of their
> load-balancing nodes. These load balancing hidden services SHOULD
> have different hidden service keys and could use stealth
> authentication for privacy (so that their introduction points are
> encrypted).
> 
> A management server would contain the actual hidden service key
> but would not need to run any hidden services. The role of the
> management server would be to regularly fetch descriptors for the
> load-balancing hidden services, combine the introduction points
> into a single descriptor for the hidden service which is then
> published as normal.
> 
> After the signature on the hidden service descriptor is verified
> there is the hidden service protocol doesn't user the permanent
> key/onion address. As such there is no need for each of the relays
> to have a copy of the hidden service key.
> 
> I think this provides a couple of advantages:
> 
> - The hidden service key only needs to be in one place. The
> machine holding the key would generate very little traffic and
> would not be locatable by the publically known hidden service
> attacks.
> 
> - The hidden service key could be stored encrypted on the
> management server.
> 
> - If any of the load balancing relays are compromised an operator 
> simply needs to stop including its introduction points in the 
> descriptor. This should minimise the need to 'revoke' hidden
> service keys.
> 
> The number of introduction points in a HS descriptor is currently 
> limited to 10 in Tor. This sets a limit on the number of
> load-balancing nodes that could be deployed at present.
> 
> This approach doesn't require any changes to the Tor code base at
> all. I hope to implement a management server in the next few weeks
> to test how this works in practice.
> 
> It would be great to get any feedback about this proposal!
> 
> Regards, Donncha
> 
> 
> 
> 
> 
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.22 (MingW32)

iQEcBAEBCAAGBQJVAcU7AAoJEIN/pSyBJlsRPfQH/2e9sdD+7Vhh1l7DMImJo7/Q
uPaU7T7pOXEbNhrzS3hvsGyPfuNK1fKuKXoLCFq8Dx+hM0S1ciyo6OJ456wqtg3E
DO1bFivBxA+/y/CvY51Kz4kssZ0sNOc4GOejw1HB7pHw8sKLzZvTKQcPJIBY+ome
5sN6Twy3y1B9rzPQNEJFJYb8FtmUOlQZvr08yzm7VCB7dJFx5cNr72m+xbTnSoRY
s+heUFmuu9lU+YOBkin+NpkmsQ32knQg1y8H4MJBgOdb59mopWah2BSI6wJC0RXy
arBjeMoP2xsLWV6qIpCKLGn1mX34q5eLLFyYf6kgcHIG94z+6/+XByywjDdZx0U=
=+dvW
-----END PGP SIGNATURE-----
-- 
tor-talk mailing list - tor-talk@xxxxxxxxxxxxxxxxxxxx
To unsubscribe or change other settings go to
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-talk