[Author Prev][Author Next][Thread Prev][Thread Next][Author Index][Thread Index]
[Libevent-users] requests over HTTPS after server-side close
Hi everybody,
based on code of the https-client sample I use libevent for doing
HTTP(S) requests in one of my applications. It receives data from
another process (via socket using libevent as well), transforms the
data, and issues HTTP requests based on the transformation result.
Between multiple data receptions from the external process there's some
random delay. In case the delay is long enough, the HTTP connection
established by libevent is closed by the server. When using HTTP
libevent automatically reconnects to the server as soon a new request
has been issued. So, everything is fine.
However, when HTTPS is used, the reconnect seems not work and no request
is performed after the server-side disconnect. I was able to reproduce
this behavior with a modified (although somewhat ugly) version of the
https-client example. You can find the source code in the attachment
(don't know if attachments are welcome in this mailing list). The
timeout in http_request_done simulates reception of data from the
external application which causes a new HTTP request to be triggered.
The timeout value should be chosen to let the callback be invoked after
the server closed the connection. You can try it with example.com and a
timeout value of 55 sec. For HTTP you will get a infinite loop of
requests. For HTTPS only one request is performed, the connection
closes, the timer callback is invoked and the application terminates.
I wonder which steps are required to get the code working with HTTPS as
it does with HTTP. From an application programmers view, I would expect
that there should be no difference, except for the bev creation (as in
the example). Do you have any suggestions? I'm using libevent 2.1.5-beta
(although I observed this in 2.1.3 and 2.1.4 as well) with OpenSSL
1.0.1k on Linux (gcc 4.8.3, glibc 2.19).
Kind regards,
Steffen
/*
This is an example of how to hook up evhttp with bufferevent_ssl
It just GETs an https URL given on the command-line and prints the response
body to stdout.
Actually, it also accepts plain http URLs to make it easy to compare http vs
https code paths.
Loosely based on le-proxy.c.
*/
// Get rid of OSX 10.7 and greater deprecation warnings.
#if defined(__APPLE__) && defined(__clang__)
#pragma clang diagnostic ignored "-Wdeprecated-declarations"
#endif
#include <stdio.h>
#include <assert.h>
#include <stdlib.h>
#include <string.h>
#include <errno.h>
#ifdef _WIN32
#include <winsock2.h>
#include <ws2tcpip.h>
#define snprintf _snprintf
#define strcasecmp _stricmp
#else
#include <sys/socket.h>
#include <netinet/in.h>
#endif
#include <event2/bufferevent_ssl.h>
#include <event2/bufferevent.h>
#include <event2/buffer.h>
#include <event2/listener.h>
#include <event2/util.h>
#include <event2/http.h>
#include <openssl/ssl.h>
#include <openssl/err.h>
#include <openssl/rand.h>
#include "openssl_hostname_validation.h"
static struct event_base *base;
static int ignore_cert = 0;
struct bufferevent *bev;
static const char* host;
static char* data_file;
static struct evhttp_connection *evcon;
static char uri[256];
static void perform_http_request(void);
static void close_cb(struct evhttp_connection *con, void *arg)
{
printf("--- connection closed ---\n");
}
static void timer_cb(evutil_socket_t fd, short what, void *arg)
{
printf("timer expired, issueing next request\n");
perform_http_request();
}
static void
http_request_done(struct evhttp_request *req, void *ctx)
{
char buffer[256];
int nread;
if (req == NULL) {
/* If req is NULL, it means an error occurred, but
* sadly we are mostly left guessing what the error
* might have been. We'll do our best... */
struct bufferevent *bev = (struct bufferevent *) ctx;
unsigned long oslerr;
int printed_err = 0;
int errcode = EVUTIL_SOCKET_ERROR();
fprintf(stderr, "some request failed - no idea which one though!\n");
/* Print out the OpenSSL error queue that libevent
* squirreled away for us, if any. */
while ((oslerr = bufferevent_get_openssl_error(bev))) {
ERR_error_string_n(oslerr, buffer, sizeof(buffer));
fprintf(stderr, "%s\n", buffer);
printed_err = 1;
}
/* If the OpenSSL error queue was empty, maybe it was a
* socket error; let's try printing that. */
if (! printed_err)
fprintf(stderr, "socket error = %s (%d)\n",
evutil_socket_error_to_string(errcode),
errcode);
return;
}
fprintf(stderr, "Response line: %d %s\n",
evhttp_request_get_response_code(req),
evhttp_request_get_response_code_line(req));
while ((nread = evbuffer_remove(evhttp_request_get_input_buffer(req),
buffer, sizeof(buffer)))
> 0) {
/* These are just arbitrary chunks of 256 bytes.
* They are not lines, so we can't treat them as such. */
fwrite(buffer, nread, 1, stdout);
}
{
struct timeval tv = { 55, 0 };
evtimer_add(evtimer_new(base, timer_cb, NULL), &tv);
}
}
static void
syntax(void)
{
fputs("Syntax:\n", stderr);
fputs(" https-client -url <https-url> [-data data-file.bin] [-ignore-cert] [-retries num]\n", stderr);
fputs("Example:\n", stderr);
fputs(" https-client -url https://ip.appspot.com/\n";, stderr);
exit(1);
}
static void
die(const char *msg)
{
fputs(msg, stderr);
exit(1);
}
static void
die_openssl(const char *func)
{
fprintf (stderr, "%s failed:\n", func);
/* This is the OpenSSL function that prints the contents of the
* error stack to the specified file handle. */
ERR_print_errors_fp (stderr);
exit(1);
}
static void perform_http_request(void)
{
int r;
struct evhttp_request *req;
struct evkeyvalq *output_headers;
struct evbuffer * output_buffer;
// Fire off the request
req = evhttp_request_new(http_request_done, bev);
if (req == NULL) {
fprintf(stderr, "evhttp_request_new() failed\n");
return;
}
output_headers = evhttp_request_get_output_headers(req);
evhttp_add_header(output_headers, "Host", host);
// evhttp_add_header(output_headers, "Connection", "close");
if (data_file) {
/* NOTE: In production code, you'd probably want to use
* evbuffer_add_file() or evbuffer_add_file_segment(), to
* avoid needless copying. */
FILE * f = fopen(data_file, "rb");
char buf[1024];
size_t s;
size_t bytes = 0;
if (!f) {
syntax();
}
output_buffer = evhttp_request_get_output_buffer(req);
while ((s = fread(buf, 1, sizeof(buf), f)) > 0) {
evbuffer_add(output_buffer, buf, s);
bytes += s;
}
evutil_snprintf(buf, sizeof(buf)-1, "%lu", (unsigned long)bytes);
evhttp_add_header(output_headers, "Content-Length", buf);
fclose(f);
}
r = evhttp_make_request(evcon, req, data_file ? EVHTTP_REQ_POST : EVHTTP_REQ_GET, uri);
if (r != 0) {
fprintf(stderr, "evhttp_make_request() failed\n");
return;
}
}
/* See http://archives.seul.org/libevent/users/Jan-2013/msg00039.html */
static int cert_verify_callback(X509_STORE_CTX *x509_ctx, void *arg)
{
char cert_str[256];
const char *host = (const char *) arg;
const char *res_str = "X509_verify_cert failed";
HostnameValidationResult res = Error;
/* This is the function that OpenSSL would call if we hadn't called
* SSL_CTX_set_cert_verify_callback(). Therefore, we are "wrapping"
* the default functionality, rather than replacing it. */
int ok_so_far = 0;
X509 *server_cert = NULL;
if (ignore_cert) {
return 1;
}
ok_so_far = X509_verify_cert(x509_ctx);
server_cert = X509_STORE_CTX_get_current_cert(x509_ctx);
if (ok_so_far) {
res = validate_hostname(host, server_cert);
switch (res) {
case MatchFound:
res_str = "MatchFound";
break;
case MatchNotFound:
res_str = "MatchNotFound";
break;
case NoSANPresent:
res_str = "NoSANPresent";
break;
case MalformedCertificate:
res_str = "MalformedCertificate";
break;
case Error:
res_str = "Error";
break;
default:
res_str = "WTF!";
break;
}
}
X509_NAME_oneline(X509_get_subject_name (server_cert),
cert_str, sizeof (cert_str));
if (res == MatchFound) {
printf("https server '%s' has this certificate, "
"which looks good to me:\n%s\n",
host, cert_str);
return 1;
} else {
printf("Got '%s' for hostname '%s' and certificate:\n%s\n",
res_str, host, cert_str);
return 0;
}
}
int
main(int argc, char **argv)
{
int r;
struct evhttp_uri *http_uri;
const char *url = NULL;
const char *scheme, *path, *query;
int port;
int retries = 0;
SSL_CTX *ssl_ctx;
SSL *ssl;
int i;
for (i = 1; i < argc; i++) {
if (!strcmp("-url", argv[i])) {
if (i < argc - 1) {
url = argv[i + 1];
} else {
syntax();
}
} else if (!strcmp("-ignore-cert", argv[i])) {
ignore_cert = 1;
} else if (!strcmp("-data", argv[i])) {
if (i < argc - 1) {
data_file = argv[i + 1];
} else {
syntax();
}
} else if (!strcmp("-retries", argv[i])) {
if (i < argc - 1) {
retries = atoi(argv[i + 1]);
} else {
syntax();
}
} else if (!strcmp("-help", argv[i])) {
syntax();
}
}
if (!url) {
syntax();
}
#ifdef _WIN32
{
WORD wVersionRequested;
WSADATA wsaData;
int err;
wVersionRequested = MAKEWORD(2, 2);
err = WSAStartup(wVersionRequested, &wsaData);
if (err != 0) {
printf("WSAStartup failed with error: %d\n", err);
return 1;
}
}
#endif // _WIN32
http_uri = evhttp_uri_parse(url);
if (http_uri == NULL) {
die("malformed url");
}
scheme = evhttp_uri_get_scheme(http_uri);
if (scheme == NULL || (strcasecmp(scheme, "https") != 0 &&
strcasecmp(scheme, "http") != 0)) {
die("url must be http or https");
}
host = evhttp_uri_get_host(http_uri);
if (host == NULL) {
die("url must have a host");
}
port = evhttp_uri_get_port(http_uri);
if (port == -1) {
port = (strcasecmp(scheme, "http") == 0) ? 80 : 443;
}
path = evhttp_uri_get_path(http_uri);
if (path == NULL) {
path = "/";
}
query = evhttp_uri_get_query(http_uri);
if (query == NULL) {
snprintf(uri, sizeof(uri) - 1, "%s", path);
} else {
snprintf(uri, sizeof(uri) - 1, "%s?%s", path, query);
}
uri[sizeof(uri) - 1] = '\0';
// Initialize OpenSSL
SSL_library_init();
ERR_load_crypto_strings();
SSL_load_error_strings();
OpenSSL_add_all_algorithms();
/* This isn't strictly necessary... OpenSSL performs RAND_poll
* automatically on first use of random number generator. */
r = RAND_poll();
if (r == 0) {
die_openssl("RAND_poll");
}
/* Create a new OpenSSL context */
ssl_ctx = SSL_CTX_new(SSLv23_method());
if (!ssl_ctx)
die_openssl("SSL_CTX_new");
#ifndef _WIN32
/* TODO: Add certificate loading on Windows as well */
/* Attempt to use the system's trusted root certificates.
* (This path is only valid for Debian-based systems.) */
if (1 != SSL_CTX_load_verify_locations(ssl_ctx,
"/etc/ssl/certs/ca-certificates.crt",
NULL))
die_openssl("SSL_CTX_load_verify_locations");
/* Ask OpenSSL to verify the server certificate. Note that this
* does NOT include verifying that the hostname is correct.
* So, by itself, this means anyone with any legitimate
* CA-issued certificate for any website, can impersonate any
* other website in the world. This is not good. See "The
* Most Dangerous Code in the World" article at
* https://crypto.stanford.edu/~dabo/pubs/abstracts/ssl-client-bugs.html
*/
SSL_CTX_set_verify(ssl_ctx, SSL_VERIFY_PEER, NULL);
/* This is how we solve the problem mentioned in the previous
* comment. We "wrap" OpenSSL's validation routine in our
* own routine, which also validates the hostname by calling
* the code provided by iSECPartners. Note that even though
* the "Everything You've Always Wanted to Know About
* Certificate Validation With OpenSSL (But Were Afraid to
* Ask)" paper from iSECPartners says very explicitly not to
* call SSL_CTX_set_cert_verify_callback (at the bottom of
* page 2), what we're doing here is safe because our
* cert_verify_callback() calls X509_verify_cert(), which is
* OpenSSL's built-in routine which would have been called if
* we hadn't set the callback. Therefore, we're just
* "wrapping" OpenSSL's routine, not replacing it. */
SSL_CTX_set_cert_verify_callback (ssl_ctx, cert_verify_callback,
(void *) host);
#endif // not _WIN32
// Create event base
base = event_base_new();
if (!base) {
perror("event_base_new()");
return 1;
}
// Create OpenSSL bufferevent and stack evhttp on top of it
ssl = SSL_new(ssl_ctx);
if (ssl == NULL) {
die_openssl("SSL_new()");
}
#ifdef SSL_CTRL_SET_TLSEXT_HOSTNAME
// Set hostname for SNI extension
SSL_set_tlsext_host_name(ssl, host);
#endif
if (strcasecmp(scheme, "http") == 0) {
bev = bufferevent_socket_new(base, -1, BEV_OPT_CLOSE_ON_FREE);
} else {
bev = bufferevent_openssl_socket_new(base, -1, ssl,
BUFFEREVENT_SSL_CONNECTING,
BEV_OPT_CLOSE_ON_FREE|BEV_OPT_DEFER_CALLBACKS);
}
if (bev == NULL) {
fprintf(stderr, "bufferevent_openssl_socket_new() failed\n");
return 1;
}
bufferevent_openssl_set_allow_dirty_shutdown(bev, 1);
// For simplicity, we let DNS resolution block. Everything else should be
// asynchronous though.
evcon = evhttp_connection_base_bufferevent_new(base, NULL, bev,
host, port);
if (evcon == NULL) {
fprintf(stderr, "evhttp_connection_base_bufferevent_new() failed\n");
return 1;
}
if (retries > 0) {
evhttp_connection_set_retries(evcon, retries);
}
evhttp_connection_set_closecb(evcon, close_cb, NULL);
perform_http_request();
event_base_dispatch(base);
evhttp_connection_free(evcon);
event_base_free(base);
#ifdef _WIN32
WSACleanup();
#endif
return 0;
}