Re: [Libevent-users] evdns behaviour questions and suggestions

[Azat Khuzhin <a3at.mail@xxxxxxxxx>]

On Thu, Jul 16, 2015 at 10:47:43PM +0500, Ilya Gordeev wrote:
> Hello. Using libevent (specifically evdns subsystem) I've encountered with
> some troubles.
> 
> 1) How should I properly interrupt all pending evdns_getaddrinfo requests
> and free evdns_base? Let's say I've caught signal to interrupt working. If I
> call evdns_getaddrinfo_cancel for all pending evdns getaddrinfo requests and
> immediately call evdns_base_free(evdns_base, 0) and event_base_free(base)
> then some memory leaks occur.

Hi Ilya,

How do you call evdns_getaddrino_cancel() for all pending events?

> In my testing program I detect memory leaks using mtrace/muntrace. (If it's
> needed I can attach testing program). In one mtrace log I've got this:
> =====
> Memory not freed:
> -----------------
>            Address     Size     Caller
> 0x00000000006035e0     0x30  at 0x7ffff799224f
> 0x0000000000603620    0x108  at 0x7ffff7992687
> 0x0000000000603730    0x108  at 0x7ffff7992687
> 0x0000000000603840     0x30  at 0x7ffff799224f
> 0x0000000000603880     0x30  at 0x7ffff799238f
> 0x0000000000603d60     0x30  at 0x7ffff799224f
> 0x0000000000603e60     0x30  at 0x7ffff799238f
> 0x0000000000603ea0     0x30  at 0x7ffff799238f
> 0x0000000000603fc0     0x30  at 0x7ffff799238f
> 0x0000000000604010    0x108  at 0x7ffff7992687
> 0x0000000000604120     0x40  at 0x7ffff7bc7496
> 0x0000000000604170     0x40  at 0x7ffff7bc7496
> 0x00000000006043e0    0x258  at 0x7ffff798d553
> 0x0000000000604640    0x108  at 0x7ffff7992687
> 0x0000000000604900    0x258  at 0x7ffff798d553
> 0x0000000000604b60    0x258  at 0x7ffff798d553
> 0x0000000000604f20    0x258  at 0x7ffff798d553
> 0x0000000000605180    0x258  at 0x7ffff798d553
> 0x00000000006056a0    0x258  at 0x7ffff798d553
> 0x0000000000605900    0x258  at 0x7ffff798d553
> =====
> Then in gdb I've got which code they are:
> =====
> (gdb) info line *0x7ffff798d553
> Line 834 of "libevent-2.1.5-beta/evdns.c" starts at address 0x7ffff798d553
> <reply_schedule_callback+35>
>    and ends at 0x7ffff798d556 <reply_schedule_callback+38>.
> (gdb) info line *0x7ffff799224f
> Line 2861 of "libevent-2.1.5-beta/evdns.c"
>    starts at address 0x7ffff799224f <evdns_base_resolve_ipv4+63> and ends at
> 0x7ffff7992252 <evdns_base_resolve_ipv4+66>.
> (gdb) info line *0x7ffff799238f
> Line 2900 of "libevent-2.1.5-beta/evdns.c"
>    starts at address 0x7ffff799238f <evdns_base_resolve_ipv6+63> and ends at
> 0x7ffff7992392 <evdns_base_resolve_ipv6+66>.
> (gdb) info line *0x7ffff7992687
> Line 4650 of "libevent-2.1.5-beta/evdns.c" starts at address 0x7ffff7992687
> <evdns_getaddrinfo+551>
>    and ends at 0x7ffff799268a <evdns_getaddrinfo+554>.
> (gdb) info line *0x7ffff7bc7496
> Line 824 of "libevent-2.1.5-beta/evutil.c" starts at address 0x7ffff7bc7496
> <evutil_new_addrinfo_+70>
>    and ends at 0x7ffff7bc7499 <evutil_new_addrinfo_+73>.
> =====

evdns uses deferred callbacks (reply_schedule_callback), and during it's
creation it allocates memory, that will be freed only by that callback
(reply_run_callback), so when request finished/timedout/e.t.c it will
create such a callback to call yours, and if you will set fail_requests=1 for
evdns_base_free() it will schedule for every pending/awaiting request
such a callback.
So even without fail_requests=1 it will create defers.

And when you call evdns_base_free() it will free pending/awaiting
requests, but not requests that finished/timedout/e.t.c.
And when you call event_base_free() it will just remove events for that
defers which is not good, since they allocated memory that will be freed
only by it's callback (but maybe I missing something).

In theory something like must works correct:
  remove_all_events_except_dns(base); // signals/rw/timers/e.t.c/bufferevents
  evdns_base_free(dns);
  event_base_dispatch(base); // <-- will call callbacks for defers, to avoid leaks
  event_base_free(base);
There could be some tricky cases, but it worth trying I think.

I think that this could be fixed by using finalizers, but need sometime
to check this.

> 2) It would be very useful in some cases if there was additional evdns
> functions similar to some event_* such as: event_base_foreach_event(),
> event_get_callback() and event_get_callback_arg().

For what? If you want to clean memory there then it is not a good
choise, because it is not always correct to just call "free(arg)".

(For example you could grep by UPCAST in evdns.c)

> 3) If evdns_getaddrinfo_cancel is called then evdns_getaddrinfo_request
> callback will be called with result = EVUTIL_EAI_CANCEL (not
> DNS_ERR_CANCEL). Am I right? If yes it was not easy to figure out, in
> documentation it is not very clear.

Not always:
- DNS_ERR_CANCEL
  This is when you call
  evdns_cancel_request()/evdns_getaddrinfo_cancel(), but not always, see
  below
- EVUTIL_EAI_CANCEL
  In case there is pending callback (i.e. there is answer from
  nameserver), then it will be called with EVUTIL_EAI_CANCEL

Regards,
Azat.
***********************************************************************
To unsubscribe, send an e-mail to majordomo@xxxxxxxxxxxxx with
unsubscribe libevent-users    in the body.