Re: [Libevent-users] Using event_active() for partial reading with bufferevents

On Thu, Nov 8, 2012 at 9:55 AM, Nick Mathewson <nickm@xxxxxxxxxxxxx> wrote:

Hello again!

On Tue, Nov 6, 2012 at 4:12 PM, Himanshu S <mail2himanshu@xxxxxxxxx> wrote:

On Tue, Nov 6, 2012 at 7:37 AM, Nick Mathewson <nickm@xxxxxxxxxxxxx> wrote:

Right now the best way to do what you have in mind is to consider *why* you only want to read partial data and then read again after the next event loop -- there's usually a better way to do whatever you're trying to achieve.

The main reason for doing this is to avoid spending unfair amount of time on a single stream.

Has this turned out to be a problem for your code in practice? Libevent already tries to round-robin its reads between multiple bufferevents by limiting the number of bytes it will read for each pass through the event loop. If you want, you can limit it even more by setting a read high-watermark so that libevent won't fill up any input buffere on a bufferevennt y more than the requested amount.

That's good to know. This solves the issue of 'reading' about fair number of bytes from all the bufferevents. But bufferevent still won't know (can't figure out), how the application is spending time on different message type. for e.g. application may have 2 message types, both of about same size, but one message may need lot more processing than another one. I am sure using watermark, and libevents inbuilt round robin mechanism would be really helpful.

But if there isn't, and you want to make sure your code works with future versions of Libevent, your best bet is either to add a feature like this in Libevent 2.1, or to create a separate event whose callback will also run the bufferevent's callback, and activate that one.

Is it true that the event_active() works only from a different thread? I see event_active() eventually calls event_callback_activate_nolock_() & it checks if the thread_id is different. So event if I create a separate event & call event_active() on that event, from the same thread, would it still work?

event_active() should work from any thread. But it won't actually get you the behavior you want: it appears that you'd want the event's callback to be invoked in the *next* iteration of the event loop, when in fact event_active() (when you call it from within a callback) is allowed to make the event active in the *current* itertion of the mainloop.

Libevent 2.1 has a notion of event_active_next() for scheduling an event to become active in the next mainloop iteration; I wonder if it's closer to what you need here. I don't know if it's exposed yet, but maybe it should be if it's good for something.

Yes. event_active_next() would definitely be useful in this case. More so, if we can use it in conjunction with bufferevents read event.

int
event_callback_activate_nolock_(struct event_base *base,
    struct event_callback *evcb)
{

...

    event_queue_insert_active(base, evcb);

    if (EVBASE_NEED_NOTIFY(base)) <-----
        evthread_notify_base(base);

This "NEED_NOTIFY" business is only meaningful when the event loop is running in another thread. To "notify" the loop means to activate an event on it (via eventfd, a socketpair, or whatever), so that it will wake up and process events. It doesn't need to happen when you're activating an event from the thread in which the loop is running.

Thanks for the insight. I will keep that in mind.

yrs,
--
Nick