[Libevent-users] Socket stuck in close_wait state

["John Jones" <johnjones1979@xxxxxxx>]

I am using libevent to comminucate between two computers on the network. The following code is a minimal example to reproduce my problem. The same code is running on both computers.

What I want to happen is that either computer can send data to the other, then close the connection. But what actually happens is that both end up with the connections hanging in the CLOSE_WAIT state and eventually the max number of open files is exceeded and everything stops.

Labelling the computers A and B, in my head this is what should be happening:

A and B both call Listen()
A calls Send() which calls bufferevent_socket_connect()
B calls Accept()
A calls HandleSendEvent() which calls bufferevent_write()
B calls Read() then bufferevent_free()
A calls Close() which calls bufferevent_free() then event_base_loopbreak()

Since bufferevent_free() is meant to close the socket I expected the socket to be closed on both sides, but my logic is obviously wrong.

It's my first time using libevent and any help would be much appreciated, thanks. Code is below.

#include <event2/listener.h>
#include <event2/bufferevent.h>
#include <event2/buffer.h>
#include <event2/util.h>
#include <event2/event.h>
#include <sys/socket.h>
#include <arpa/inet.h>
#include <stdlib.h>
#include <stdio.h>
#include <string.h>
#include <iostream>
#include <thread>

void HandleReadEvent( bufferevent *bev, short events, void *arg )
{
        if ( events & ( BEV_EVENT_EOF | BEV_EVENT_ERROR ) ) 
        {
            bufferevent_free( bev );
        }
}
 
void Read( bufferevent *bev, void *arg )
{
        /* This callback is invoked when there is data to read on bev. */
                 
        unsigned char buf[10];
    unsigned long n;
    evbuffer *input = bufferevent_get_input( bev );    
    while ( ( n = evbuffer_get_length( input ) ) > 0 ) 
    {
        n = evbuffer_remove( input, buf, sizeof( buf ) );
    }
 
     bufferevent_free( bev );    
}
 
void Accept( evconnlistener *listener, evutil_socket_t sock, sockaddr *address, int socklen, void *arg )
{
        /* We got a new connection! Set up a bufferevent for it. */
        event_base *base = evconnlistener_get_base( listener );
        bufferevent *bev = bufferevent_socket_new( base, sock, BEV_OPT_CLOSE_ON_FREE );
 
        bufferevent_setcb( bev, Read, NULL, HandleReadEvent, NULL);
 
        bufferevent_enable( bev, EV_READ|EV_WRITE );
}
 
void Listen()
{
    event_base *tcpbase;
    evconnlistener *tcplistener;
    sockaddr_in tcpsin;
         
    memset( &tcpsin, 0, sizeof( tcpsin ) );
    tcpsin.sin_family = AF_INET;
    tcpsin.sin_addr.s_addr = htonl( INADDR_ANY );
    tcpsin.sin_port = htons( 55000 );
         
    tcpbase = event_base_new();
         
     tcplistener = evconnlistener_new_bind( tcpbase, Accept, NULL,  LEV_OPT_CLOSE_ON_FREE|LEV_OPT_REUSEABLE, -1, (struct  sockaddr*)&tcpsin, sizeof(tcpsin) );
        if ( !tcplistener ) 
        {
            printf( "Couldn't create tcp listener\n" );
            return;
        }
     
    /* Enter the event loop; does not return. */
    event_base_dispatch( tcpbase );        
}
 
void Close( bufferevent *bev, void *arg )
{
    bufferevent_free( bev );
    event_base_loopbreak( bufferevent_get_base( bev ) );
}
 
void HandleSendEvent( bufferevent *bev, short events, void *arg )
{
    if ( events & BEV_EVENT_CONNECTED ) 
    {
        bufferevent_setcb( bev, NULL, Close, HandleSendEvent, arg );
          
        char data[10] = {'0','1','2','3','4','5','6','7','8','9'};
          
        bufferevent_write( bev, data, 10 );
    } 
    else if ( events & ( BEV_EVENT_EOF | BEV_EVENT_ERROR ) ) 
    {
        Close( bev, arg );
    }
}
 
void Send()
{
    event_base *tcpbase;
    bufferevent *bev;
    sockaddr_in tcpsin;
     
    tcpbase = event_base_new();
     
    memset( &tcpsin, 0, sizeof(tcpsin) );
    tcpsin.sin_family = AF_INET;    
     
    tcpsin.sin_port = htons( 55000 );
    bev = bufferevent_socket_new( tcpbase, -1, BEV_OPT_CLOSE_ON_FREE );
     
    inet_pton( AF_INET, "192.168.20.3", &(tcpsin.sin_addr) );    
         
    bufferevent_setcb( bev, NULL, NULL, HandleSendEvent, NULL );
     
    bufferevent_enable( bev, EV_READ|EV_WRITE );
     
    if ( bufferevent_socket_connect( bev, (struct sockaddr *)&tcpsin, sizeof(tcpsin) ) < 0 ) 
    {
        bufferevent_free( bev );
    }
     
    event_base_dispatch( tcpbase );
}
 
int main( int argc, char* argv[] )
{
    std::thread listen (Listen);
     
    while ( true )
    {
        int c;
        std::cin >> c;
        Send();
    }
     
    listen.join();
     
    return 0;
}

***********************************************************************
To unsubscribe, send an e-mail to majordomo@xxxxxxxxxxxxx with
unsubscribe libevent-users    in the body.