Re: Parallelizing (some of) Tor's AES Operations

[Nick Mathewson <nickm@xxxxxxxxxxxxx>]

On Fri, May 02, 2008 at 12:05:11AM -0400, Matt Edman wrote:
  [...]
> > Alternatively, you could give each circuit _two_ cell queues for each
> > direction: one for cells that need to be crypted, and one for cells
> > that are already crypted and are ready to transmit.  All you'd need to
> > tell the workers about is which circuits have to be processed; all
> > you'd need to tell the main thread about is which circuits now have
> > more data.  I like this approach a little better because it doesn't
> > require quite so much bookkeeping to tell the workers about what keys
> > to use to crypt what, or to tell the main thread where to put cells.
> 
> I like this approach better too. Separate cell queues definitely seems like
> the way to go.
> 
> > It will definitely take some thinking to figure out exactly what kind
> > of locking and notification mechanisms you'd want to use here.  If you
> > have any questions about waking the main thread, or what kind of
> > integrity Tor's data structures need, just ask.
> 
> In terms of locking and notification mechanisms, is it reasonable to assume a
> threaded environment? I notice CPU workers are typically threads, but may be
> fork()ed processes on some platforms. If TOR_IS_MULTITHREADED is not defined,
> though, we don't get to use the handy locking constructs that already exist in
> compat.h. Are there really many platforms that don't handle threads very well?

There are a couple.  IMO, for the timeframe and use-cases you're
thinking of, I'd suggest you write your code with the assumption that
TOR_IS_MULTITHREADED is always defined.  On platforms without good
threading, we can just not have separate threads to handle AES.

> It also looks like compat.h already has wrappers for thread
> signaling, but are just '#if 0'ed out right now. I assume that's
> simply because they're not currently used anywhere in Tor though
> (rather than because it's broken).

Right.  Be aware however that they're as-yet untested; use at your own
risk.

yrs,
-- 
Nick