Re: [Libevent-users] Valgrind, evmap_io_add Invalid write of size 4

[Catalin Patulea <catalinp@xxxxxxxxxx>]

I saw similar things a long while back, stacks looking like this:

==18923== Invalid write of size 8
==18923==    at 0x452C2C: evmap_io_add (evmap.c:328)
==18923==    by 0x446715: event_add_internal (event.c:2033)
==18923==    by 0x446AF3: event_add (event.c:1926)
==18923==    by 0x44E91C: _bufferevent_add_event (bufferevent.c:860)
==18923==    by 0x44FE66: be_socket_enable (bufferevent_sock.c:552)
==18923==    by 0x44EF47: bufferevent_enable (bufferevent.c:422)
==18923==    by 0x45B7BC: evhttp_connection_connect (http.c:2215)
==18923==    by 0x45B945: evhttp_make_request (http.c:2270)
[...]

==18923== Invalid write of size 8
==18923==    at 0x4524D0: evmap_io_del (evmap.c:384)
==18923==    by 0x4471C3: event_del (event.c:2211)
==18923==    by 0x44F85D: be_socket_disable (bufferevent_sock.c:569)
==18923==    by 0x44E286: bufferevent_suspend_write (bufferevent.c:96)
==18923==    by 0x450198: bufferevent_socket_connect_hostname
(bufferevent_sock.c:487)
==18923==    by 0x45B7E0: evhttp_connection_connect (http.c:2217)
==18923==    by 0x45B945: evhttp_make_request (http.c:2270)
[...]

There's definitely some state sync issues between what evhttp thinks
about a socket, and what evmap/epoll think.

At the time, this patch reduced our crash rate significantly (7x):
http://archives.seul.org/libevent/users/Dec-2012/msg00034.html

(thread continued here:
http://archives.seul.org/libevent/users/Feb-2013/msg00020.html)

Nick, you proposed a different variant of the patch in this post:
http://archives.seul.org/libevent/users/Feb-2013/msg00028.html

But I never had a chance to try it.

Sorry I can't offer more concrete information.

On Mon, Jun 16, 2014 at 8:50 AM, Nick Mathewson <nickm@xxxxxxxxxxxxx> wrote:
> On Mon, Jun 16, 2014 at 8:42 AM, Nick Mathewson <nickm@xxxxxxxxxxxxx> wrote:
>> On Sun, Jun 15, 2014 at 10:28 AM, Robin <imer@xxxxxxx> wrote:
>>>
>>  [...]
>>
>>> I've just tried it with 2.0.21 (or were you talking about 2.1.x?) and it's
>>> still happening.
>>>>
>>>> ==14175== Invalid write of size 4
>
> I assume you're compiling for a 32-bit architecture here.  If not, my
> analysis below is wrong.
>
>>>> ==14175==    at 0x5348E76: evmap_io_add (evmap.c:328)
>>
>> If I'm matching the line number to the version properly, line 328 is:
>
> Whoa, sorry, premature send. :(
>
> If I'm matching the line number to the version properly, line 328 is:
>         TAILQ_INSERT_TAIL(&ctx->events, ev, ev_io_next);
>
> Is that what you have?  (In other words, if you match the failing line
> number to the code you compiled, is that the line you're seeing?)
>
> The TAILQ_INSERT_TAIL macro is defined as:
>
> #define TAILQ_INSERT_TAIL(head, elm, field) do { \
>         (elm)->field.tqe_next = NULL; \
>         (elm)->field.tqe_prev = (head)->tqh_last; \
>         *(head)->tqh_last = (elm); \
>         (head)->tqh_last = &(elm)->field.tqe_next; \
> } while (0)
>
> So the only pointers accessed there are fields in ev->ev_io_next,
> fields in ctx->events, and (indirectly) whatever the tqh_last pointer
> points to.
>
> I think it's unlikely that "ev->ev_io_next" or "ctx->events" would be
> freed or invalid, since they're part of the ev and ctx structures
> respectively, and those structures are used elsewhere in
> evmap_io_add().
>
> What seems more likely to me is that the ctx->events.tqh_last point
> has become corrupt, and no longer points to a valid event pointer.
> The most common way for this to happen would be if one of the events
> for this fd has been freed without first removing it from the
> event_base with event_del().  It might also happen, I think, if you
> use event_assign() or event_set() to change an event's configuration
> without first removing it from the event_base with event_del().
>
> hope this helps,
> --
> Nick
> ***********************************************************************
> To unsubscribe, send an e-mail to majordomo@xxxxxxxxxxxxx with
> unsubscribe libevent-users    in the body.
***********************************************************************
To unsubscribe, send an e-mail to majordomo@xxxxxxxxxxxxx with
unsubscribe libevent-users    in the body.