[Author Prev][Author Next][Thread Prev][Thread Next][Author Index][Thread Index]

Re: [Libevent-users] Problems with deferred HTTP handlers over SSL



Hi Oscar,

Thanks for following up.

We tried building and linking with the same versions of libevent, libevhtp as static libraries, and openssl as a shared library as you did on our Linux machines and still got the same errors. (Unfortunately, we didn't get any "hello"s.)

The rate of error might be platform specific, but it seems like this test, when built on Mac, dies much faster than 10k invocations.

$ while [ true ]; do curl --insecure https://127.0.0.1:1025/; done
hellohellohellohellohellohellohellohellohellohellohellohellohellohellohellohellohellohellohellohellohellohellohellohellohellohellohellohellohellohellohellohellohellohellohellohellohellohellohellohellohellohellohellohellohellohellohellocurl:
(56) SSL: unable to obtain common name from peer certificate
hellohellohellohellohellohellohellohellohellohellohellohellohellohellohellohellohellohellohellohellohellohellohellohellohellohellohellohellohellohellohellohellohellohellohellohellohellohellohellohellohellohellohellohellohellohellohellohellohellohellohellohellohellohellohellohellohellohellohellohellohellohellohellohellohellohellohellohellohellohellohellohellohellohellohellocurl:
(56) SSL: unable to obtain common name from peer certificate
curl: (35) Unknown SSL protocol error in connection to 127.0.0.1:1025
curl: (7) couldn't connect to host
curl: (7) couldn't connect to host

I'm attaching the config.h file for our libevent installation on Linux as well, in case it is useful.

Not only that, we just discovered that this error still happens even for normal HTTP requests on Linux -- by commenting out "evhtp_ssl_init(http, &scfg);" -- but takes a little longer.

With the non-SSL case, but thread still detached, the server dies with the following error.
[err] event_queue_remove: 0x16cdbc8(fd 27) not on queue 8

So maybe this is a problem for both SSL and non-SSL, but it manifests itself quicker with SSL probably with something to do with timing? It's just a guess.

As for evhtp_set_gencb, it's only there to make the test code simpler.

Thanks.

On Fri, Jan 13, 2012 at 3:53 PM, Oscar Koeroo <okoeroo@xxxxxxxxx> wrote:
Hi Amarin,

I've retried it by looping curl and it indeed seg. faults after about 10k
invocations (just a guestimate). I did remove the sleep(1); to make it crash
faster, it doesn't really matter.

BTW: why did you use the evhtp_set_gencb()?
Example:
https://github.com/okoeroo/lcmaps-rest/blob/master/src/lcmapsd_httprest.c


       Oscar


On 13/1/12 9:20 PM, Oscar Koeroo wrote:
> Hi Amarin,
>
> Could you checkout the "0.4.5" tag of libevhtp and rebuild with that one? I
> don't think master is the right starting point.
>
> I tried your code and my Chrome (version 18.x devel), Firefox 9.0.1 and
> Opera 11.60 worked fine.
>
> I've statically build your code against libevent release-2.0.16-stable and
> libevhtp 0.4.5 and dynamically linked it with OpenSSL 0.9.8 on my Mac.
>
> In friendly letters all my browsers say "hello".
>
>
>       Oscar
>
>
> On 13/1/12 8:50 PM, Amarin Phaosawasdi wrote:
>> Hi All,
>>
>> We're working on fixing an issue with one of our servers, which serves
>> HTTPS requests on one end, and makes asynchronous RPCs on the other end.
>>
>> The server uses libevhtp to serve HTTPS requests. But since it is also
>> based on libevent and we suspect it might have something to do with
>> bufferevent_openssl, we thought it is appropriate to post it here.
>>
>> When a HTTPS request comes in, a the server invokes an async RPC to an
>> internal server. When the RPC finishes, we then use the RPC results to
>> provide an HTTPS response to the browser.
>>
>> However, almost all the time the browser gives an error.
>> - "SSL received a record with an incorrect Message Authentication Code.
>> (Error code: ssl_error_bad_mac_read)." (in Firefox)
>> - "Error 107 (net::ERR_SSL_PROTOCOL_ERROR): SSL protocol error." (in Chrome)
>> - "Error 324 (net::ERR_EMPTY_RESPONSE): The server closed the connection
>> without sending any data. (in Chrome)
>>
>> Rarely, when the browser doesn't give an error, it either succeeds or
>> blocks forever.
>>
>> Repeated fails normally end with a segmentation fault in the server.
>>
>> When a synchronous RPC is used instead (everything is finished on the
>> callback stack), the error is gone and every request is successful.
>>
>> Also if the server is configured to serve HTTP requests instead of HTTPS,
>> it also always succeeds regardless or the RPC calling mode.
>>
>> Does anybody have an idea what might be going on?
>>
>> The attached code can be used to reproduce this issue. When the server is
>> run, use a browser to connect to https://127.0.0.1:1025.
>>
>> Tested with:
>> gcc/g++ 4.4.5
>> Both openssl 0.9.8 and 1.0.0f
>> libevhtp development branch
>> libevent master branch
>> Debian 2.6.39-bpo.2-amd64 x86_64
>>
>> Thank you,
>> Amarin
>>
>
>



/* config.h.  Generated from config.h.in by configure.  */
/* config.h.in.  Generated from configure.in by autoheader.  */

/* Define if libevent should build without support for a debug mode */
/* #undef DISABLE_DEBUG_MODE */

/* Define if libevent should not allow replacing the mm functions */
/* #undef DISABLE_MM_REPLACEMENT */

/* Define if libevent should not be compiled with thread support */
/* #undef DISABLE_THREAD_SUPPORT */

/* Define to 1 if you have the `arc4random' function. */
/* #undef HAVE_ARC4RANDOM */

/* Define to 1 if you have the `arc4random_buf' function. */
/* #undef HAVE_ARC4RANDOM_BUF */

/* Define to 1 if you have the <arpa/inet.h> header file. */
#define HAVE_ARPA_INET_H 1

/* Define to 1 if you have the `clock_gettime' function. */
#define HAVE_CLOCK_GETTIME 1

/* Define to 1 if you have the declaration of `CTL_KERN', and to 0 if you
   don't. */
#define HAVE_DECL_CTL_KERN 1

/* Define to 1 if you have the declaration of `KERN_ARND', and to 0 if you
   don't. */
#define HAVE_DECL_KERN_ARND 0

/* Define to 1 if you have the declaration of `KERN_RANDOM', and to 0 if you
   don't. */
#define HAVE_DECL_KERN_RANDOM 1

/* Define to 1 if you have the declaration of `RANDOM_UUID', and to 0 if you
   don't. */
#define HAVE_DECL_RANDOM_UUID 1

/* Define if /dev/poll is available */
/* #undef HAVE_DEVPOLL */

/* Define to 1 if you have the <dlfcn.h> header file. */
#define HAVE_DLFCN_H 1

/* Define if your system supports the epoll system calls */
#define HAVE_EPOLL 1

/* Define to 1 if you have the `epoll_ctl' function. */
#define HAVE_EPOLL_CTL 1

/* Define to 1 if you have the `eventfd' function. */
#define HAVE_EVENTFD 1

/* Define if your system supports event ports */
/* #undef HAVE_EVENT_PORTS */

/* Define to 1 if you have the `fcntl' function. */
#define HAVE_FCNTL 1

/* Define to 1 if you have the <fcntl.h> header file. */
#define HAVE_FCNTL_H 1

/* Define to 1 if the system has the type `fd_mask'. */
#define HAVE_FD_MASK 1

/* Do we have getaddrinfo()? */
#define HAVE_GETADDRINFO 1

/* Define to 1 if you have the `getegid' function. */
#define HAVE_GETEGID 1

/* Define to 1 if you have the `geteuid' function. */
#define HAVE_GETEUID 1

/* Define this if you have any gethostbyname_r() */
/* #undef HAVE_GETHOSTBYNAME_R */

/* Define this if gethostbyname_r takes 3 arguments */
/* #undef HAVE_GETHOSTBYNAME_R_3_ARG */

/* Define this if gethostbyname_r takes 5 arguments */
/* #undef HAVE_GETHOSTBYNAME_R_5_ARG */

/* Define this if gethostbyname_r takes 6 arguments */
/* #undef HAVE_GETHOSTBYNAME_R_6_ARG */

/* Define to 1 if you have the `getnameinfo' function. */
#define HAVE_GETNAMEINFO 1

/* Define to 1 if you have the `getprotobynumber' function. */
#define HAVE_GETPROTOBYNUMBER 1

/* Define to 1 if you have the `getservbyname' function. */
/* #undef HAVE_GETSERVBYNAME */

/* Define to 1 if you have the `gettimeofday' function. */
#define HAVE_GETTIMEOFDAY 1

/* Define to 1 if you have the `inet_aton' function. */
#define HAVE_INET_ATON 1

/* Define to 1 if you have the `inet_ntop' function. */
#define HAVE_INET_NTOP 1

/* Define to 1 if you have the `inet_pton' function. */
#define HAVE_INET_PTON 1

/* Define to 1 if you have the <inttypes.h> header file. */
#define HAVE_INTTYPES_H 1

/* Define to 1 if you have the `issetugid' function. */
/* #undef HAVE_ISSETUGID */

/* Define to 1 if you have the `kqueue' function. */
/* #undef HAVE_KQUEUE */

/* Define if the system has zlib */
#define HAVE_LIBZ 1

/* Define to 1 if you have the <memory.h> header file. */
#define HAVE_MEMORY_H 1

/* Define to 1 if you have the `mmap' function. */
#define HAVE_MMAP 1

/* Define to 1 if you have the <netdb.h> header file. */
#define HAVE_NETDB_H 1

/* Define to 1 if you have the <netinet/in6.h> header file. */
/* #undef HAVE_NETINET_IN6_H */

/* Define to 1 if you have the <netinet/in.h> header file. */
#define HAVE_NETINET_IN_H 1

/* Define if the system has openssl */
#define HAVE_OPENSSL 1

/* Define to 1 if you have the <openssl/bio.h> header file. */
#define HAVE_OPENSSL_BIO_H 1

/* Define to 1 if you have the `pipe' function. */
#define HAVE_PIPE 1

/* Define to 1 if you have the `poll' function. */
#define HAVE_POLL 1

/* Define to 1 if you have the <poll.h> header file. */
#define HAVE_POLL_H 1

/* Define to 1 if you have the `port_create' function. */
/* #undef HAVE_PORT_CREATE */

/* Define to 1 if you have the <port.h> header file. */
/* #undef HAVE_PORT_H */

/* Define if you have POSIX threads libraries and header files. */
/* #undef HAVE_PTHREAD */

/* Define if we have pthreads on this system */
#define HAVE_PTHREADS 1

/* Define to 1 if you have the `putenv' function. */
#define HAVE_PUTENV 1

/* Define to 1 if the system has the type `sa_family_t'. */
#define HAVE_SA_FAMILY_T 1

/* Define to 1 if you have the `select' function. */
#define HAVE_SELECT 1

/* Define to 1 if you have the `sendfile' function. */
#define HAVE_SENDFILE 1

/* Define to 1 if you have the `setenv' function. */
#define HAVE_SETENV 1

/* Define if F_SETFD is defined in <fcntl.h> */
#define HAVE_SETFD 1

/* Define to 1 if you have the `sigaction' function. */
#define HAVE_SIGACTION 1

/* Define to 1 if you have the `signal' function. */
#define HAVE_SIGNAL 1

/* Define to 1 if you have the `splice' function. */
#define HAVE_SPLICE 1

/* Define to 1 if you have the <stdarg.h> header file. */
#define HAVE_STDARG_H 1

/* Define to 1 if you have the <stddef.h> header file. */
#define HAVE_STDDEF_H 1

/* Define to 1 if you have the <stdint.h> header file. */
#define HAVE_STDINT_H 1

/* Define to 1 if you have the <stdlib.h> header file. */
#define HAVE_STDLIB_H 1

/* Define to 1 if you have the <strings.h> header file. */
#define HAVE_STRINGS_H 1

/* Define to 1 if you have the <string.h> header file. */
#define HAVE_STRING_H 1

/* Define to 1 if you have the `strlcpy' function. */
/* #undef HAVE_STRLCPY */

/* Define to 1 if you have the `strsep' function. */
#define HAVE_STRSEP 1

/* Define to 1 if you have the `strtok_r' function. */
#define HAVE_STRTOK_R 1

/* Define to 1 if you have the `strtoll' function. */
#define HAVE_STRTOLL 1

/* Define to 1 if the system has the type `struct addrinfo'. */
#define HAVE_STRUCT_ADDRINFO 1

/* Define to 1 if the system has the type `struct in6_addr'. */
#define HAVE_STRUCT_IN6_ADDR 1

/* Define to 1 if `s6_addr16' is a member of `struct in6_addr'. */
#define HAVE_STRUCT_IN6_ADDR_S6_ADDR16 1

/* Define to 1 if `s6_addr32' is a member of `struct in6_addr'. */
#define HAVE_STRUCT_IN6_ADDR_S6_ADDR32 1

/* Define to 1 if the system has the type `struct sockaddr_in6'. */
#define HAVE_STRUCT_SOCKADDR_IN6 1

/* Define to 1 if `sin6_len' is a member of `struct sockaddr_in6'. */
/* #undef HAVE_STRUCT_SOCKADDR_IN6_SIN6_LEN */

/* Define to 1 if `sin_len' is a member of `struct sockaddr_in'. */
/* #undef HAVE_STRUCT_SOCKADDR_IN_SIN_LEN */

/* Define to 1 if the system has the type `struct sockaddr_storage'. */
#define HAVE_STRUCT_SOCKADDR_STORAGE 1

/* Define to 1 if `ss_family' is a member of `struct sockaddr_storage'. */
#define HAVE_STRUCT_SOCKADDR_STORAGE_SS_FAMILY 1

/* Define to 1 if `__ss_family' is a member of `struct sockaddr_storage'. */
/* #undef HAVE_STRUCT_SOCKADDR_STORAGE___SS_FAMILY */

/* Define to 1 if you have the <sys/devpoll.h> header file. */
/* #undef HAVE_SYS_DEVPOLL_H */

/* Define to 1 if you have the <sys/epoll.h> header file. */
#define HAVE_SYS_EPOLL_H 1

/* Define to 1 if you have the <sys/eventfd.h> header file. */
#define HAVE_SYS_EVENTFD_H 1

/* Define to 1 if you have the <sys/event.h> header file. */
/* #undef HAVE_SYS_EVENT_H */

/* Define to 1 if you have the <sys/ioctl.h> header file. */
#define HAVE_SYS_IOCTL_H 1

/* Define to 1 if you have the <sys/mman.h> header file. */
#define HAVE_SYS_MMAN_H 1

/* Define to 1 if you have the <sys/param.h> header file. */
#define HAVE_SYS_PARAM_H 1

/* Define to 1 if you have the <sys/queue.h> header file. */
#define HAVE_SYS_QUEUE_H 1

/* Define to 1 if you have the <sys/select.h> header file. */
#define HAVE_SYS_SELECT_H 1

/* Define to 1 if you have the <sys/sendfile.h> header file. */
#define HAVE_SYS_SENDFILE_H 1

/* Define to 1 if you have the <sys/socket.h> header file. */
#define HAVE_SYS_SOCKET_H 1

/* Define to 1 if you have the <sys/stat.h> header file. */
#define HAVE_SYS_STAT_H 1

/* Define to 1 if you have the <sys/sysctl.h> header file. */
#define HAVE_SYS_SYSCTL_H 1

/* Define to 1 if you have the <sys/time.h> header file. */
#define HAVE_SYS_TIME_H 1

/* Define to 1 if you have the <sys/types.h> header file. */
#define HAVE_SYS_TYPES_H 1

/* Define to 1 if you have the <sys/uio.h> header file. */
#define HAVE_SYS_UIO_H 1

/* Define to 1 if you have the <sys/wait.h> header file. */
#define HAVE_SYS_WAIT_H 1

/* Define if TAILQ_FOREACH is defined in <sys/queue.h> */
#define HAVE_TAILQFOREACH 1

/* Define if timeradd is defined in <sys/time.h> */
#define HAVE_TIMERADD 1

/* Define if timerclear is defined in <sys/time.h> */
#define HAVE_TIMERCLEAR 1

/* Define if timercmp is defined in <sys/time.h> */
#define HAVE_TIMERCMP 1

/* Define if timerisset is defined in <sys/time.h> */
#define HAVE_TIMERISSET 1

/* Define to 1 if the system has the type `uint16_t'. */
#define HAVE_UINT16_T 1

/* Define to 1 if the system has the type `uint32_t'. */
#define HAVE_UINT32_T 1

/* Define to 1 if the system has the type `uint64_t'. */
#define HAVE_UINT64_T 1

/* Define to 1 if the system has the type `uint8_t'. */
#define HAVE_UINT8_T 1

/* Define to 1 if the system has the type `uintptr_t'. */
#define HAVE_UINTPTR_T 1

/* Define to 1 if you have the <unistd.h> header file. */
#define HAVE_UNISTD_H 1

/* Define to 1 if you have the `unsetenv' function. */
#define HAVE_UNSETENV 1

/* Define to 1 if you have the `vasprintf' function. */
#define HAVE_VASPRINTF 1

/* Define if kqueue works correctly with pipes */
/* #undef HAVE_WORKING_KQUEUE */

/* Define to 1 if you have the <zlib.h> header file. */
#define HAVE_ZLIB_H 1

/* Define to the sub-directory in which libtool stores uninstalled libraries.
   */
#define LT_OBJDIR ".libs/"

/* Define to 1 if your C compiler doesn't accept -c and -o together. */
/* #undef NO_MINUS_C_MINUS_O */

/* Numeric representation of the version */
#define NUMERIC_VERSION 0x02001000

/* Name of package */
#define PACKAGE "libevent"

/* Define to the address where bug reports for this package should be sent. */
#define PACKAGE_BUGREPORT ""

/* Define to the full name of this package. */
#define PACKAGE_NAME ""

/* Define to the full name and version of this package. */
#define PACKAGE_STRING ""

/* Define to the one symbol short name of this package. */
#define PACKAGE_TARNAME ""

/* Define to the home page for this package. */
#define PACKAGE_URL ""

/* Define to the version of this package. */
#define PACKAGE_VERSION ""

/* Define to necessary symbol if this constant uses a non-standard name on
   your system. */
/* #undef PTHREAD_CREATE_JOINABLE */

/* The size of `int', as computed by sizeof. */
#define SIZEOF_INT 4

/* The size of `long', as computed by sizeof. */
#define SIZEOF_LONG 8

/* The size of `long long', as computed by sizeof. */
#define SIZEOF_LONG_LONG 8

/* The size of `pthread_t', as computed by sizeof. */
#define SIZEOF_PTHREAD_T 8

/* The size of `short', as computed by sizeof. */
#define SIZEOF_SHORT 2

/* The size of `size_t', as computed by sizeof. */
#define SIZEOF_SIZE_T 8

/* The size of `void *', as computed by sizeof. */
#define SIZEOF_VOID_P 8

/* Define to 1 if you have the ANSI C header files. */
#define STDC_HEADERS 1

/* Define to 1 if you can safely include both <sys/time.h> and <time.h>. */
#define TIME_WITH_SYS_TIME 1

/* Version number of package */
#define VERSION "2.0.16-stable"

/* Define to appropriate substitue if compiler doesnt have __func__ */
/* #undef __func__ */

/* Define to empty if `const' does not conform to ANSI C. */
/* #undef const */

/* Define to `__inline__' or `__inline' if that's what the C compiler
   calls it, or to nothing if 'inline' is not supported under any name.  */
#ifndef __cplusplus
/* #undef inline */
#endif

/* Define to `int' if <sys/types.h> does not define. */
/* #undef pid_t */

/* Define to `unsigned int' if <sys/types.h> does not define. */
/* #undef size_t */

/* Define to unsigned int if you dont have it */
/* #undef socklen_t */

/* Define to `int' if <sys/types.h> does not define. */
/* #undef ssize_t */