Re: [Libevent-users] requests over HTTPS after server-side close

[Mark Ellzey <mthomas@xxxxxxxxxx>]

On Sat, Jan 24, 2015 at 05:33:44PM +0100, Steffen Christgau wrote:
> Hi everybody,
> 
> based on code of the https-client sample I use libevent for doing
> HTTP(S) requests in one of my applications. It receives data from
> another process (via socket using libevent as well), transforms the
> data, and issues HTTP requests based on the transformation result.
> Between multiple data receptions from the external process there's some
> random delay. In case the delay is long enough, the HTTP connection
> established by libevent is closed by the server. When using HTTP
> libevent automatically reconnects to the server as soon a new request
> has been issued. So, everything is fine.
> 
> However, when HTTPS is used, the reconnect seems not work and no request
> is performed after the server-side disconnect. I was able to reproduce
> this behavior with a modified (although somewhat ugly) version of the
> https-client example. You can find the source code in the attachment

Here be dragons!

Welcome to the world of proxyies. Many think it's an easy task to
tackle, but you ran into one of the many hundreds of edge cases that
will soak up all of your time.

SInce you're using beta, and I am strongly opposed to the new design of
bufferevent_openssl, mainly because I ran into many problems with it.
I think it was work done based on some dicussions we had on IRC, but I
never realized it was implemented until the other day, when I ran it and
thought "this doesn't work as well as intended". I honestly think the
BIO method should be rolled back. It's just exibits strange behaviour.


> (don't know if attachments are welcome in this mailing list). The
> timeout in http_request_done simulates reception of data from the
> external application which causes a new HTTP request to be triggered.
> The timeout value should be chosen to let the callback be invoked after
> the server closed the connection. You can try it with example.com and a
> timeout value of 55 sec. For HTTP you will get a infinite loop of
> requests. For HTTPS only one request is performed, the connection
> closes, the timer callback is invoked and the application terminates.
> 

One of the ways I dealt with this issue was to keep 3 queues:

pending   - this queue contains a list of requests to be sent 
active    - this queue contains a list of reuquests which are being
            processed
ready     - a list of backend servers which are established and waiting
            for new requests.

I also have 3 states for the connections to the backend servers:

active - this connection is being used
idle   - this connection is idle and available for use
down   - connection is down, it will continue trying to reconnect 
         every so osften. 

When a new request comes in: insert_into_pending_queue(bufferevent_accept(...));

When a backend server transitions from active/down to idle, it will emit an event 
that calls a function which does something like the following:

-----
for each pending request:
    backend = find_idle_backend_connection()
    
	  backend_connection_set_active(backend);
-----

If for some reason the backend is down or unabe to process the request,
you simply unset the EV_READ flags, and plop it back into the pending
queue.

If the request is finished, then: backend_connection_set_idle(backend)
 
Or as the main comment in my code states upon getting a new connection:

    /* This function is called whenever evhtp has matched a hostname on a request.
     *
     * Once the downstream request has been initialized and setup, this function
     * will *NOT* immediately start processing the request. This is because a
     * downstream connection may not be available at the time this function is
     * called.
     *
     * Instead, the request is placed in a pending request queue where this
     * queue is processed once downstream_connection_set_idle() has signaled the
     * process_pending event handler that a downstream connection is available.
     *
     * The return value of this function, EVHTP_RES_PAUSE, informs the evhtp
     * backend to suspend reading on the socket on the upstream until
     * process_pending successfully finds an idle downstream connection. From
     * there the upstream request is resumed.
     *
     * TL;DNR: upstream request is not immediately processed, but placed in a
     *         pending queue until a downstream connection becomes available.
     */

> I wonder which steps are required to get the code working with HTTPS as
> it does with HTTP. From an application programmers view, I would expect

For what you are trying to do, other than "retries" you're going to run into many
other problems, but you will learn that as you go continue to write
whatever it is you're writing..

> that there should be no difference, except for the bev creation (as in
> the example). Do you have any suggestions? I'm using libevent 2.1.5-beta
> (although I observed this in 2.1.3 and 2.1.4 as well) with OpenSSL
> 1.0.1k on Linux (gcc 4.8.3, glibc 2.19).
> 
> Kind regards,

My only suggestions for simplicity sake is to setup your eventcb's to
catch connections being closed prematurely on both ends. Possibly using
a hybrid of the teqchiqus I use above.
***********************************************************************
To unsubscribe, send an e-mail to majordomo@xxxxxxxxxxxxx with
unsubscribe libevent-users    in the body.