Re: [Libevent-users] Multithreading

[Nick Mathewson <nickm@xxxxxxxxxxxxx>]

On Tue, Apr 10, 2012 at 6:41 PM, Jan Danielsson
<jan.m.danielsson@xxxxxxxxx> wrote:
> Hello,
>
>   I'm not really clear on what EV_WRITE means for UDP. Does it
> essentially guarantee that sendto() won't return EAGAIN, ENOBUFS et al
> (backend quirks aside)?

Basically, yes.

>   Assuming I'm on a pthreads platform, and I have a few TCP channels
> which are used for query-operations, and a query can take a while
> (anything from under a second to a few minutes), is it safe to hand over
> a "struct bufferevent *ev" to other thread, and let it handle the reply?

It should be, so long as you've configured threading correctly in the
event base, and made the bufferevent threadsafe when you created it.

> I.e:
>
>   ----------------------------
>   void do_read(struct bufferevent *bev, void *ctx)
>   {
>     parsedata *ps = ctx;
>     int done;
>
>     done = parse_query(bev, ps);
>
>     if(done)
>     {
>       start_worker_thread(worker, bev, ps);
>     }
>   }
>
>   void worker(struct bufferevent *bev, parsedata *ps)
>   {
>      struct evbuffer *output = bufferevent_get_output(bev);
>      char *result;
>      int len;
>
>      long_operation(ps, &result, &len);
>
>      evbuffer_add(output, result, len)
>   }
>   ----------------------------
>
>   How does one handle the situation where a client closes the socket
> prematurely (during long_operation() for instance)? Does 'bev' and
> 'output' become toxic [by the base thread] if the client closes the
> socket, or will they exist until I call event_del() on bev? (Which I
> would assume is safe to do on the worker() thread, given I'm running
> libevent 2.x?).

If the client closes the socket, you should find out about it by
getting a BEV_EVENT_EOF on the bufferevent's eventcb  callback.

The client can close the *TCP connection*, but only you can close the socket.

There's no such thing as calling event_del() on a bev; I'm assuming
you mean bufferevent_disable() or bufferevent_free() or something.

Obviously, if you call bufferevent_free() on a bufferevent, you
shouldn't be adding data to it thereafter, so you'll need some way to
synchronize between the threads.

>   With regards to multithreading and UDP: Is there any point to queuing
> outbound UDP packets, and send them in a "do_write()" event handler,
> rather than just calling sendto() directly from the worker threads which
> generate outbound packets? I.e does this:
>   ----------------------------
>   workerthread(...)
>   {
>     packet_t *pkt;
>     ...
>     make_packet(&pkt);
>     lock(outqueue)
>     queue_push(outqueue, pkt);
>     unlock(outqueue)
>     if(!ev)
>       ev = event_new(base, s, EV_WRITE, do_write, NULL);
>   }

(You probably wanted an event_add here, and you probably wanted to add
EV_PERSIST to that list of flags.)

>   do_write(...)
>   {
>      ...
>      lock(outqueue)
>      pkt = queue_pull(outqueue)
>      unlock(outqueue)
>      if(pkt)
>        sendto(..., pkt->data, ...)

(You probably want to detect EAGAIN on the sendto.)

>      else
>      {
>        event_del(ev)
>        ev = NULL
>      }
>   }
>   ----------------------------
>
>   [the handling of ev is merely conceptual, the point is that the event
> handler is added when there are packets to be send, and it is removed
> when there are no more packets waiting to be transferred]
>
>   ...offer any any benefits over this:
>
>   ----------------------------
>   workerthread(...)
>   {
>     packet_t *pkt;
>     ...
>     make_packet(&pkt);
>     sendto(..., pkt->data, ...)
>   }
>   ----------------------------
>
>   ?

I'm not seeing a great reason to prefer the first approach; it'll
depend on what circumstances there are for which your IP stack gives
EAGAIN when doing a UDP write.  Then again, I'm not as versed on UDP
behavior as I'd like to be, so please don't take me too seriously on
this one.

peace,
-- 
Nick
***********************************************************************
To unsubscribe, send an e-mail to majordomo@xxxxxxxxxxxxx with
unsubscribe libevent-users    in the body.