Re: Re: Re: [Libevent-users] Assert fail in evhttp_connection_fail_ with req=0x00 on error=EVREQ_HTTP_TIMEOUT at http.c:691

[gerkenjohannes@xxxxxx]

Thank you for your reply.
Â
The patch works for my "https-client testcase".
Â
I now have the following problem in my program:
Â

[err] event_assign called on an already added event 0x7fffe8a29d50 (events: 0x52, fd: 20, flags: 0x83)
[warn] Epoll ADD(5) on fd 20 failed. Old events were 0; read change was 1 (add); write change was 1 (add); close change was 0 (none): Bad file descriptor
Program received signal SIGABRT, Aborted.
[Switching to Thread 0x7fffef72e700 (LWP 31129)]
0x00007ffff4c33967 in raise () from /usr/lib/libc.so.6
(gdb) bt full
#0Â 0x00007ffff4c33967 in raise () from /usr/lib/libc.so.6
No symbol table info available.
#1Â 0x00007ffff4c34d3a in abort () from /usr/lib/libc.so.6
No symbol table info available.
#2Â 0x00007ffff7bab0b5 in event_exit (errcode=-559030611) at log.c:105
No locals.
#3Â 0x00007ffff7bab4a2 in event_errx (eval=-559030611, fmt=0x7ffff7bc7a60 "%s called on an already added event %p (events: 0x%x, fd: %d, flags: 0x%x)") at log.c:162
ÂÂÂÂÂÂÂ ap = {{gp_offset = 48, fp_offset = 48, overflow_arg_area = 0x7fffef72dc28, reg_save_area = 0x7fffef72db60}}
#4Â 0x00007ffff7b9c783 in event_assign (ev=0x7fffe8a29d50, base=0xeda2c0, fd=-1, events=82, callback=0x7ffff57d0ae4 <be_openssl_handshakeeventcb>, arg=0x7fffe8a29d40) at event.c:2002
ÂÂÂÂÂÂÂ dent = 0x7fffe8a5d520
ÂÂÂÂÂÂÂ find = {node = {hte_next = 0x0}, ptr = 0x7fffe8a29d50, added = 0}
ÂÂÂÂÂÂÂ __func__ = "event_assign"
#5Â 0x00007ffff57d0c4b in set_handshake_callbacks (bev_ssl=0x7fffe8a29d40, fd=-1) at bufferevent_openssl.c:1083
ÂÂÂÂÂÂÂ bev = 0x7fffe8a29d40
ÂÂÂÂÂÂÂ r1 = 0
ÂÂÂÂÂÂÂ r2 = 0
#6Â 0x00007ffff57d143b in be_openssl_ctrl (bev=0x7fffe8a29d40, op=BEV_CTRL_SET_FD, data=0x7fffef72dd40) at bufferevent_openssl.c:1284
ÂÂÂÂÂÂÂ bev_ssl = 0x7fffe8a29d40
#7Â 0x00007ffff7b917d2 in bufferevent_setfd (bev=0x7fffe8a29d40, fd=-1) at bufferevent.c:850
ÂÂÂÂÂÂÂ d = {ptr = 0xffffffff, fd = -1}
ÂÂÂÂÂÂÂ res = -1
#8Â 0x00007ffff7bc1b04 in evhttp_connection_connect_ (evcon=0x7fffe8d1a3c0) at http.c:2410
ÂÂÂÂÂÂÂ old_state = 0
ÂÂÂÂÂÂÂ __func__ = "evhttp_connection_connect_"
#9Â 0x00007ffff7bc1e0c in evhttp_make_request (evcon=0x7fffe8d1a3c0, req=0x7fffe8cd5c00, type=EVHTTP_REQ_GET, uri=0x7fffe8a65000 "/") at http.c:2482
ÂÂÂÂÂÂÂ res = 0
ÂÂÂÂÂÂÂ __func__ = "evhttp_make_request"
#10 0x000000000040cb80 in M_HTTP_main_thread () at ../M_HTTP.cpp:311
ÂÂÂÂÂÂÂ i = 6
ÂÂÂÂÂÂÂ _http_cmd = 0x0
ÂÂÂÂÂÂÂ threadLock = {m = 0xe92ce8, is_locked = true}
#11 0x00000000004174c1 in boost::detail::thread_data<void (*)()>::run (this=0xe92380) at /usr/include/boost/thread/detail/thread.hpp:115
No locals.
#12 0x00007ffff747fdca in ?? () from /usr/lib/libboost_thread.so.1.56.0
No symbol table info available.
#13 0x00007ffff69d1314 in start_thread () from /usr/lib/libpthread.so.0
No symbol table info available.
#14 0x00007ffff4ce83ed in clone () from /usr/lib/libc.so.6
No symbol table info available.
Â
Â
If I try to prevent this, then I get multiple "Bad file descriptor" messages.
Â

Gesendet:ÂSonntag, 21. September 2014 um 21:50 Uhr
Von:Â"Azat Khuzhin" <a3at.mail@xxxxxxxxx>
An:Âlibevent-users@xxxxxxxxxxxxx
Betreff:ÂRe: Re: [Libevent-users] Assert fail in evhttp_connection_fail_ with req=0x00 on error=EVREQ_HTTP_TIMEOUT at http.c:691
On Fri, Sep 19, 2014 at 02:40:43PM +0200, gerkenjohannes@xxxxxx wrote:
> I'm using the "master" from github.
>
> The problem only occurs, when I set a timeout/retry for an https connection(to an server with port 443 closed). The segfault/assert-fail occurs, if I use EVENT_BASE_FLAG_EPOLL_USE_CHANGELIST in combination with the timeouts/retrys.
>
> Testcase:
> add the following to sample/https-client.c
>
> struct timeval retry_timeout;
> retry_timeout.tv_sec=5;
> retry_timeout.tv_usec=0;
> evhttp_connection_set_timeout(evcon,15);
> evhttp_connection_set_retries(evcon,3);
> evhttp_connection_set_initial_retry_tv(evcon,&retry_timeout);
>
> execute: https_client -url https://libevent.org
>
> output:
>
> [warn] Epoll ADD(1) on fd 6 failed. Old events were 0; read change was 1 (add); write change was 0 (none); close change was 0 (none): Bad file descriptor
> [debug] event_add: event: 0x25a5b30 (fd 6), EV_WRITE EV_TIMEOUT call 0x7f1942897ae4
> [warn] Epoll ADD(4) on fd 6 failed. Old events were 0; read change was 0 (none); write change was 1 (add); close change was 0 (none): Bad file descriptor
> [debug] event_process_active: event_callback 0x25a5c40, closure 3, call 0x7f1941fc8879
> some request failed - no idea which one though!
> socket error = Bad file descriptor (9)

After digging through syscalls traces, I got the next patch, could you test it
please:
From 6dc4de3005bda27ca1154e6fa2abd323c6d4de44 Mon Sep 17 00:00:00 2001
From: Azat Khuzhin <a3at.mail@xxxxxxxxx>
Date: Sun, 21 Sep 2014 18:29:17 +0400
Subject: [PATCH] bufferevent_openssl: reset fd_is_set when setfd with -1 is
called

Before this patch:
$ sample/https-client -retries 1 -url https://libevent.org[https://libevent.org]
[warn] Epoll ADD(1) on fd 6 failed. Old events were 0; read change was 1 (add); write change was 0 (none); close change was 0 (none): Bad file descriptor
[warn] Epoll ADD(4) on fd 6 failed. Old events were 0; read change was 0 (none); write change was 1 (add); close change was 0 (none): Bad file descriptor
some request failed - no idea which one though!
socket error = Bad file descriptor (9)

After this patch:
$ sample/https-client -retries 1 -url https://libevent.org[https://libevent.org]
Response line: 0 (null)

Reported-by: gerkenjohannes@xxxxxx
---
bufferevent_openssl.c | 2 ++
1 file changed, 2 insertions(+)

diff --git a/bufferevent_openssl.c b/bufferevent_openssl.c
index 423b34f10097..b30f90ff2e94 100644
--- a/bufferevent_openssl.c
+++ b/bufferevent_openssl.c
@@ -1276,6 +1276,8 @@ be_openssl_ctrl(struct bufferevent *bev,
SSL_set_bio(bev_ssl->ssl, bio, bio);
bev_ssl->fd_is_set = 1;
}
+ if (data->fd == -1)
+ bev_ssl->fd_is_set = 0;
if (bev_ssl->state == BUFFEREVENT_SSL_OPEN)
return set_open_callbacks(bev_ssl, data->fd);
else {
--
2.1.0

Full version of this patch is here:
https://github.com/azat/libevent/commit/3da84c2949cc693bb6e6a65a3b1de48af1be0483.patch[https://github.com/azat/libevent/commit/3da84c2949cc693bb6e6a65a3b1de48af1be0483.patch]

Cheers, Azat.

> Â
>
> Gesendet:ÂDonnerstag, 04. September 2014 um 09:44 Uhr
> Von:Â"Azat Khuzhin" <a3at.mail@xxxxxxxxx>
> An:Âlibevent-users@xxxxxxxxxxxxx
> Cc:Âlibevent-users@xxxxxxxx
> Betreff:ÂRe: [Libevent-users] Assert fail in evhttp_connection_fail_ with req=0x00 on error=EVREQ_HTTP_TIMEOUT at http.c:691
> On Fri, Aug 29, 2014 at 06:30:26PM +0200, gerkenjohannes@xxxxxx wrote:
> > Hi,
> >
> > when I add new evhttp requests to an running event_loop, I get "epoll ADD: Bad file descriptor" and later "SIGABRT".
> > The event_base is running in an seperate worker thread, the main thread is doing the evhttp_make_request.
> >
> > Output from GDB:
> > http://paste.ubuntu.com/8176676/[http://paste.ubuntu.com/8176676/][http://paste.ubuntu.com/8176676/[http://paste.ubuntu.com/8176676/]]
> >
> > Output from GDB with libevent debugging on:
> > http://paste.ubuntu.com/8176681/[http://paste.ubuntu.com/8176681/][http://paste.ubuntu.com/8176681/[http://paste.ubuntu.com/8176681/]]
> >
> > I get the assert fail in the folowing 2 cases:
> >
> > case 1:
> > 1. add new requests to the running event_base
> >
> > case 2:
> > 1. call loop_brake from the main thread
> > 2. wait for the loop to "brake" in the worker thread
> > 3. put the worker thread to sleep
> > 4. add new requests
> > 5. wake up the worker thread
> > 6. run the event_loop
> >
> > Is this bug in libevent or my code?
>
> Which version of libevent do you have?
> Could you produce simple program or regress test for http (inside
> libevent), to test it?
***********************************************************************
To unsubscribe, send an e-mail to majordomo@xxxxxxxxxxxxx with
unsubscribe libevent-users in the body.
***********************************************************************
To unsubscribe, send an e-mail to majordomo@xxxxxxxxxxxxx with
unsubscribe libevent-users    in the body.