Re: [Libevent-users] evbuffer_add_file() file descriptor exhaustion experiment

[Azat Khuzhin <a3at.mail@xxxxxxxxx>]

On Sun, Nov 02, 2014 at 11:10:34PM +0100, Marcin Szewczyk wrote:
> On Sun, Nov 02, 2014 at 10:40:08PM +0300, Azat Khuzhin wrote:
> > On Fri, Oct 31, 2014 at 02:52:43PM +0100, Marcin Szewczyk wrote:
> > > I am struggling with exhaustion of file descriptors when using
> > > evbuffer_add_file(). My application responds to every line of input with
> > > contents of a file. I don't know how to efficiently control open
> > > descriptors number so when input rate is high I hit the open file
> 
> > And what you want to do when you are out of fds?
> > (Actually you already know this - open(2) will return -1 and errno will
> > be installed to EMFILE)
> 
> Actually the final goal is to keep a pool of free fds for new
> connections and suspend evbuffer_add_file() responses.
> 
> > *But* I guess you could use evbuffer_file_segment*() API, it have all
> > you need - cleanup callback, so you could track when some of fds closed,
> > and since you are already could track when some of fds opened (you call
> > open(2)), you will have 'open file descriptors count' implementation.

Also you could free some data from output buffers I guess, to release
some fds, a simple patch for you sample attached:

--- /tmp/evbuffer_add_file.c.orig	2014-11-03 01:58:55.153487950 +0300
+++ /tmp/evbuffer_add_file.c	2014-11-03 02:04:46.950474010 +0300
@@ -21,6 +21,7 @@
 #include <string.h>
 #include <error.h>
 #include <errno.h>
+#include <assert.h>
 #include <signal.h>
 #include <event2/event.h>
 #include <event2/bufferevent.h>
@@ -105,10 +106,14 @@
     while ((line = evbuffer_readln (ctx->evbuff_in, &len, EVBUFFER_EOL_CRLF))) {
         logmsg ("got line (output evbuffer length: %ld", evbuffer_get_length (ctx->evbuff_out));
         fd = open (ctx->fpath, O_RDONLY);
-        if (fd < 0)
-            error (1, errno, "open");
+        if (fd < 0) {
+            evbuffer_unfreeze (ctx->evbuff_out, 1);
+            assert(!evbuffer_drain (ctx->evbuff_out, evbuffer_get_length(ctx->evbuff_out)));
+            fd = open (ctx->fpath, O_RDONLY);
+        }
+        assert (fd >= 0);
         fstat (fd, &stat_);
-        evbuffer_add_file (ctx->evbuff_out, fd, 0, stat_.st_size);
+        assert(!evbuffer_add_file (ctx->evbuff_out, fd, 0, stat_.st_size));
     }
 }
 

But you need to save source files for this buffers before draining them to
allow to respond clients after awhile.
And it is more complicated then just don't trying to add to this buffers when
program are out of fds.

Cheers,
Azat.

> > 
> > Is this what you want?
> 
> Yes. Now I feel embarrassed. Somehow I was so focused on libevent 2.0
> that I missed callbacks for segments in 2.1.
> 
> > P.S. evbuffer_add_file() do use evbuffer_file_segment*() API, but
> > without any cleanup callback and with EVBUF_FS_CLOSE_ON_FREE flag.
> 
> Thank you very much for pulling me out from brain dead-lock.
***********************************************************************
To unsubscribe, send an e-mail to majordomo@xxxxxxxxxxxxx with
unsubscribe libevent-users    in the body.