Re: [Libevent-users] Problems with deferred HTTP handlers over SSL

["Hochhaus, Andrew" <ahochhaus@xxxxxxxxxxxx>]

Thanks for the help Mark.

On Sun, Jan 15, 2012 at 12:30 AM, Mark Ellzey <mthomas@xxxxxxxxxx> wrote:
> evhtp_use_threads() is a connection pooler, each thread having its own
> event_base, and each connection being accepted to that threads evbase.
>
> in this case here, you are double threading, thus you break out of evhtp
> and back into thread safety in libevent itself.
>
> Libevent needs to be setup as thread-safe too if it is to be used this
> way via one of the evthread_use_* functions.

I modified Amarin's test program to call evthread_use_pthreads()
[attached]. I also added BEV_OPT_THREADSAFE to the
bufferevent_openssl_socket_new/bufferevent_socket_new calls in
evhtp.c.

This improves things, however, I now see that the calls to
evhtp_send_reply in DeferredHandler do not return. Possibly internal
deadlock?

Additionally, about 1 in 5 requests never display in the browser but
instead wait forever (likely related to evhtp_send_reply not
returning).

Thanks,
-Andy

#include <unistd.h>
#include <pthread.h>
#include "evhtp.h"

void *DeferredHandler(void *arg) {
  sleep(1);
  evhtp_request_t *req = (evhtp_request_t*)arg;
  evhtp_headers_add_header(
      req->headers_out,
      evhtp_header_new("content-type", "text/plain; charset=utf-8", 0, 0));
  evbuffer_add(req->buffer_out, "hello", 5);
  printf("DeferredHandler -- before send_reply\n");
  evhtp_send_reply(req, EVHTP_RES_OK);
  printf("DeferredHandler -- after send_reply\n");
  pthread_exit(NULL);
}

void Handler(evhtp_request_t *req, void *) {
  pthread_t t;
  pthread_create(&t, NULL, DeferredHandler, (void*)req);
  pthread_detach(t);  // this fails
  // pthread_join(t, NULL);  // but this works
}

int main() {
  if (evthread_use_pthreads() != 0) {
    return 1;
  }

  evbase_t *base = event_base_new();
  if (!base) return 1;

  evhtp_t *http = evhtp_new(base, NULL);
  if (!http) return 1;

  evhtp_set_gencb(http, Handler, NULL);
  evhtp_use_threads(http, NULL, 4, NULL);

  evhtp_ssl_cfg_t scfg;
  char ciphers[] = "RC4+RSA:HIGH:+MEDIUM:+LOW";
  char libevhtp_pemfile[] = "libevhtp.pem";
  scfg.pemfile = libevhtp_pemfile;
  char libevhtp_privfile[] = "libevhtp.key";
  scfg.privfile = libevhtp_privfile;
  scfg.cafile = NULL;
  scfg.capath = NULL;
  scfg.ciphers = ciphers;
  scfg.ssl_opts = SSL_OP_NO_SSLv2;
  scfg.verify_peer = SSL_VERIFY_NONE;
  scfg.verify_depth = 0;
  scfg.x509_verify_cb = NULL;
  scfg.x509_chk_issued_cb = NULL;
  scfg.store_flags = 0;
  scfg.scache_type = evhtp_ssl_scache_type_internal;
  scfg.scache_timeout = 1024;
  scfg.scache_size = 1024;
  scfg.scache_init = NULL;
  scfg.scache_add = NULL;
  scfg.scache_get = NULL;
  scfg.scache_del = NULL;
  scfg.args = NULL;
  evhtp_ssl_init(http, &scfg);

  // bind and serve
  if (evhtp_bind_socket(http, "127.0.0.1", 1025, 128) == -1) return 1;
  event_base_loop(base, 0);
  event_base_free(base);

  return 0;
}