Re: [Libevent-users] How to activate read callback when not all data was read

[deepak jain <deepakjain111@xxxxxxxxx>]

Hi,

I was having similar kind of problem.

in order to resolve it, I stopped processing my commands in read callback. I read commands in read callback and pushed them to my synchronized queue where threads are waiting on that queue for commands to process them. This approach reduced burden on read callback. It s just a typical producer consumer case where read callback is producer and threads are consumer.

Any suggestion on this approach is most welcome.

DJ

On Tue, Apr 10, 2012 at 9:03 PM, Nick Mathewson <nickm@xxxxxxxxxxxxx> wrote:

On Tue, Apr 10, 2012 at 5:10 AM, Nils Rennebarth
<nils.rennebarth@xxxxxxxxx> wrote:
> Hi,
>
> In my daemon which uses libevent, i use bufferevent to read client commands
> from a socket, set everything up to eventually generate the reply, and return to
> the event loop.
>
> Now a client may send several commands in one go. If I only read the first
> command and drain the corresponding bytes from the bufferevent's input
> buffer, the read callback will not be called until *more* data arrives,
> so if a client sends two commands in one go, I need to somehow queue
> the second (and further) commands before returning to the main loop,
> otherwise the commands never gets processed.
>
> Is there some other way, to get the second command processed,
> something like "return to the main loop, but mark this callback as
> still pending, and call it again"

There isn't a way to cause this this right now; most bufferevent users
wind up doing something like this in their read handlers:

   while (there is a command on the inbuf) {
     remove command;
     process command;
   }

or something like sticking bufferevents that need more handling into a
queue, and processing them with an idle handler, like you note.

That said, I'm not at all opposed to adding a way to do this in 2.1,
assuming it's a reasonably elegant interface.  Would other people find
this useful?  What should the interface look like?

(As a sidenote, the other week, in the branch "21_event_callback" on
my github repository, I started doing exploratory work to try to
refactor the way that event callbacks are handled, made active, and so
on.  I was originally doing this to try to simplify some code, fix
some bufferevent_openssl bugs, and make us able to add support for
Chris Davis's hybrid-loop code on windows, but I think it might be
applicable here too when it's done.)

> In other words, bufferevent forces me to do edge-triggered event
> handling. Is there a way to let me do level-triggered event handling
> instead.
>
> Or is there something like a idle task, that get called when there is
> nothing else to do? Working with timeouts would introduce arbitrary gaps
> in command handling, event if there are no other clients that want
> work, so I won't go that route. Or does setting a timeout of { 0, 0 }
> work?

Vincent Bernat is right that setting a low priority here is crucial if
you want it to work like a real idle timer.  As another piece of
advice: you don't need to use timeouts at all!  It is faster to simply
make the event and activate it by hand.

For example, you don't need to say:

  struct timeval no_time = {0,0};
  ev = event_new(base, -1, 0, process_work_queue_cb, NULL);
  event_add(ev, &no_time};

Instead you can simply say:

  ev = event_new(base, -1, 0, process_work_queue_cb, NULL);
  event_active(ev, EV_TIMEOUT, 1);

The second approach is better because it doesn't require you to put
the event in the timeout heap at all: instead, Libevent sees that
you're activating the event, and puts it right in the queue of active
events.

hope this helps,
--
Nick

***********************************************************************
To unsubscribe, send an e-mail to majordomo@xxxxxxxxxxxxx with
unsubscribe libevent-users    in the body.