Re: [tor-talk] Load Balancing/High Availability Hidden Services

[benjamin barber <barberb@xxxxxxxxxxx>]

I myself have been making solar powered servers running tor, and have built
a CMS vi-chan derivative which publishes .torrents for the bittorrent
maelstrom browser.

On Thu, Mar 12, 2015 at 9:56 AM, s7r <s7r@xxxxxxxxxx> wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
>
> Hi Donncha,
>
> Your idea is indeed very interesting. I am eager to see it in
> practice. If you have the time, me and TheCthulhu (cc'ed) could
> provide the infrastructure and necessary system administration hours
> to test this.
>
> While your current solution is a way to use load balancing and
> redundancy, which means spreading the resource usage over multiple
> machines, checkout my questions about maximizing a single HS server as
> close as possible to its total capacity:
>
> https://lists.torproject.org/pipermail/tor-talk/2015-March/037173.html
>
> Please email directly if you want to start this, we will document each
> step and share the results with everyone here. Maybe this will give
> some hints for writing a proposal for load balancing and high capacity
> hidden services which could be merged with the proposal of second
> generation hidden services.
>
> On 3/12/2015 3:22 PM, Donncha O'Cearbhaill wrote:
> >
> >
> > On 11/03/15 17:40, George Kadianakis wrote:
> >> MacLemon <tor@xxxxxxxxxxx> writes:
> >>
> >>> Hoi!
> >>>
> >>> I'm looking into ideas for creating âload balancedâ or âhigh
> >>> availabilityâ hidden services. Mostly pertaining to web servers
> >>> serving large-ish static files. (Let's say 5-100MB each.)
> >>>
> >>> Load balanced as in not all requests end up at the same box to
> >>> speed up downloads. High availability as in the service is
> >>> still available if one box goes down or is taken offline for
> >>> maintenance.
> >>>
> >>> So, not exactly your usual distributed-cluster setup.
> >>>
> >>>
> >>> From what I understand it would not make sense to run the same
> >>> HS Key on multiple boxes since the descriptors would overwrite
> >>> each other every few minutes.
> >>>
> >>> I don't think one can do something like Round-Robin DNS with
> >>> HS.
> >>>
> >>> So the only way I can imagine this to work is a central
> >>> redirection node that know about all the nodes and more or less
> >>> intelligently/randomly 302 redirects each file request to a
> >>> known-to-it server.
> >>>
> >>> This still leaves a single-point-of-failure in form of the
> >>> redirection server but would at least distribute the traffic
> >>> load across multiple servers and cope for nodes coming and
> >>> going.
> >>>
> >>> Has anyone done something like this?
> >>>
> >>
> >> An application-layer load balancer like HAProxy might be able to
> >> help you.
> >>
> >> Unfortunately, there is not something equivalent to DNS Round
> >> Robin for hidden services yet. There are some ideas on how to do
> >> this on the Introduction Point-layer, but a proposal still needs
> >> to be written. For further reading:
> >> https://lists.torproject.org/pipermail/tor-dev/2014-April/006788.html
> >>
> >>
> https://lists.torproject.org/pipermail/tor-dev/2014-May/006812.html
> >>
> >
> > I've been thinking a little about this problem. It seems like one
> > simple solution would be to find a way of combining the
> > descriptors/introduction points from multiple Tor HS instances into
> > one hidden service descriptor from which the client can pick intro
> > points at random.
> >
> > To implement such a solution like that, there needs to be a means
> > for the hidden service instances to communicate with other. They
> > can then either selected a common set of intro points or combine
> > the individual sets of intro points selected by each instance.
> >
> > In one straight forward implementation a hidden service operator
> > could set up a Tor relay and a hidden service on each of their
> > load-balancing nodes. These load balancing hidden services SHOULD
> > have different hidden service keys and could use stealth
> > authentication for privacy (so that their introduction points are
> > encrypted).
> >
> > A management server would contain the actual hidden service key
> > but would not need to run any hidden services. The role of the
> > management server would be to regularly fetch descriptors for the
> > load-balancing hidden services, combine the introduction points
> > into a single descriptor for the hidden service which is then
> > published as normal.
> >
> > After the signature on the hidden service descriptor is verified
> > there is the hidden service protocol doesn't user the permanent
> > key/onion address. As such there is no need for each of the relays
> > to have a copy of the hidden service key.
> >
> > I think this provides a couple of advantages:
> >
> > - The hidden service key only needs to be in one place. The
> > machine holding the key would generate very little traffic and
> > would not be locatable by the publically known hidden service
> > attacks.
> >
> > - The hidden service key could be stored encrypted on the
> > management server.
> >
> > - If any of the load balancing relays are compromised an operator
> > simply needs to stop including its introduction points in the
> > descriptor. This should minimise the need to 'revoke' hidden
> > service keys.
> >
> > The number of introduction points in a HS descriptor is currently
> > limited to 10 in Tor. This sets a limit on the number of
> > load-balancing nodes that could be deployed at present.
> >
> > This approach doesn't require any changes to the Tor code base at
> > all. I hope to implement a management server in the next few weeks
> > to test how this works in practice.
> >
> > It would be great to get any feedback about this proposal!
> >
> > Regards, Donncha
> >
> >
> >
> >
> >
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v2.0.22 (MingW32)
>
> iQEcBAEBCAAGBQJVAcU7AAoJEIN/pSyBJlsRPfQH/2e9sdD+7Vhh1l7DMImJo7/Q
> uPaU7T7pOXEbNhrzS3hvsGyPfuNK1fKuKXoLCFq8Dx+hM0S1ciyo6OJ456wqtg3E
> DO1bFivBxA+/y/CvY51Kz4kssZ0sNOc4GOejw1HB7pHw8sKLzZvTKQcPJIBY+ome
> 5sN6Twy3y1B9rzPQNEJFJYb8FtmUOlQZvr08yzm7VCB7dJFx5cNr72m+xbTnSoRY
> s+heUFmuu9lU+YOBkin+NpkmsQ32knQg1y8H4MJBgOdb59mopWah2BSI6wJC0RXy
> arBjeMoP2xsLWV6qIpCKLGn1mX34q5eLLFyYf6kgcHIG94z+6/+XByywjDdZx0U=
> =+dvW
> -----END PGP SIGNATURE-----
> --
> tor-talk mailing list - tor-talk@xxxxxxxxxxxxxxxxxxxx
> To unsubscribe or change other settings go to
> https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-talk
>
-- 
tor-talk mailing list - tor-talk@xxxxxxxxxxxxxxxxxxxx
To unsubscribe or change other settings go to
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-talk