Re: [Libevent-users] Help with progress thread

[Ralph Castain <rhc@xxxxxxxxxxxx>]

On Nov 9, 2010, at 9:03 PM, Nick Mathewson wrote:

> On Tue, Nov 9, 2010 at 3:28 PM, Ralph Castain <rhc@xxxxxxxxxxxx> wrote:
>> Hi folks
>> 
>> I'm running into a problem that probably results from my ignorance. So I figured I would ask if someone can tell me what I'm doing wrong.
>> 
>> I have a thread that loops the event library with the following call:
>> 
>>        events += event_loop(mca_oob_tcp_component.event_base, EVLOOP_ONCE);
>> 
>> In another thread, I create and add an event that is supposed to occur when a file descriptor is ready for "write":
>> 
>>         event_set(mca_oob_tcp_component.event_base,
>>                       &peer->peer_send_event,
>>                       peer->peer_sd,
>>                       EV_WRITE|EV_PERSIST,
>>                       mca_oob_tcp_peer_send_handler,
>>                       peer);
>>         event_add(&peer->peer_send_event, 0);
>> 
>> Note that the event_set and event_add can (and in this case, probably do) occur while I am in the event_loop function.
>> 
>> What I find is that this event never gets fired. However, events that are set and added -before- going into the loop do fire.
>> 
>> So my question is: given that this is a dynamic system, how do I get the event_loop to "see" new events? I want to block in the loop, so I don't really want to set NONBLOCK if I can avoid it (otherwise, I would have to add another blocking call somewhere to keep the thread from endlessly cycling).
> 
> So I'm assuming that you've got all the threading callbacks set up
> (probably via evthread_use_pthreads()) before you created the event
> base, so that evthread_make_base_notifiable() was called on the
> event_base when you created it.  If that's not the case, that's
> probably the problem there.

I missed that - this may well be my problem. Let me dig into this a little and get back to you.

FWIW: I don't see myself drop out of the event loop when I add the event. In fact, just the opposite - I'm stuck in the loop and can't get out. So I'm not sure the bug mentioned below is accurate.

Let me see what happens when I setup the event base for notification.

Thanks!

> 
> Otherwise, I think this one is an honest-to-goodness bug.
> 
> Here's what happens when you add an event from another thread.
> 
> The event_add locks the event_base's structures, makes the changes
> necessary to add the event, and then notices that it isn't running in
> the main thread, so it needs to "alert" the main thread to exit and
> re-enter the dispatch function[*].  It does this by calling
> evthread_notify_base(), which sends a byte down a pipe (on most Unix)
> or a socketpair (on Windows), or by using an eventfd (on Linux) [**].
> There's an (EVLIST_INTERNAL) event handler for this
> pipe/socketpair/eventfd that notices that we've been "notified" so we
> can drain the pipe/socketpair/eventfd.
> 
> Now here's where I think the bug is: that event counts as an active
> event, and processing it gets counted as processing an event, so if
> you're running the loop with EVLOOP_ONCE, the loop will exit right
> after it notices that it should wake up, even though no user callbacks
> were actually entered.
> 
> The logic is in event_base_loop():
> 
>    		if (N_ACTIVE_CALLBACKS(base)) {
> 			event_process_active(base);
> 			if (!base->event_count_active && (flags & EVLOOP_ONCE))
> 				done = 1;
>                } ...
> 
> I think that the second "if" might be wrong.  We maybe want to
> continue looping not only if there are active callbacks, but also if
> the only active events that we processed were internal events.
> 
> With the current behavior, with EVLOOP_ONCE, I think adding the event
> from another thread will exit the loop right away.  Is that consistent
> with the behavior you're seeing?  If not, then the bug is possibly in
> the notification stuff, though that part actually is fairly well
> tested.
> 
> It's also possible that I'm writing this too late at night for me, and
> I've deeply misanalyzed the code here.
> 
> (As an aside, we should probably also continue looping if
> N_ACTIVE_CALLBACKS(base) is nonzero.)
> 
> [*] For backends like poll and select, we need to re-enter the
> dispatch function after getting event changes so that we can call
> select/poll with their new arguments.  For stuff like epoll and
> kqueue, we need to process the changelist and re-enter the wait
> function.
> [**] Yes, I do indeed think that kqueue should use EVFILT_USER, but
> that's a 2.1 thing.
> 
> -- 
> Nick
> ***********************************************************************
> To unsubscribe, send an e-mail to majordomo@xxxxxxxxxxxxx with
> unsubscribe libevent-users    in the body.

***********************************************************************
To unsubscribe, send an e-mail to majordomo@xxxxxxxxxxxxx with
unsubscribe libevent-users    in the body.