[Author Prev][Author Next][Thread Prev][Thread Next][Author Index][Thread Index]

Re: [Libevent-users] Different behavior of my code

To: libevent-users@xxxxxxxxxxxxx
Subject: Re: [Libevent-users] Different behavior of my code
From: Nicholas Marriott <nicholas.marriott@xxxxxxxxx>
Date: Thu, 8 Sep 2011 16:46:45 +0100
Delivered-to: archiver@xxxxxxxx
Delivered-to: libevent-users-outgoing@xxxxxxxx
Delivered-to: libevent-users@xxxxxxxx
Delivery-date: Thu, 08 Sep 2011 11:47:00 -0400
In-reply-to: <20110908154426.GU11356@xxxxxxxxxxxxxxxxxx>
References: <4E600508.8020601@xxxxxxxxx> <20110906102507.GI11356@xxxxxxxxxxxxxxxxxx> <CAN3LtdOTGA=hq6Po8zRc3jgqTpY9qV6eUdPEjpdfaUPdznSepA@xxxxxxxxxxxxxx> <20110908154426.GU11356@xxxxxxxxxxxxxxxxxx>
Reply-to: libevent-users@xxxxxxxxxxxxx
Sender: owner-libevent-users@xxxxxxxxxxxxx
User-agent: Mutt/1.5.20 (2009-06-14)

It is however true to say that files will always return as readable and
writable but that still means kqueue and poll and hence libevent works
:-).


On Thu, Sep 08, 2011 at 04:44:26PM +0100, Nicholas Marriott wrote:
> On Thu, Sep 08, 2011 at 01:17:22AM -0700, Scott Lamb wrote:
> > On Tue, Sep 6, 2011 at 3:25 AM, Nicholas Marriott
> > <nicholas.marriott@xxxxxxxxx> wrote:
> > > OpenBSD is libevent 1.4.13.
> > >
> > > OS X polling mechanisms are pretty bad: their poll() and kqueue() only
> > > support sockets, so you will need to use select (set EVENT_NOPOLL=1
> > > EVENT_NOKQUEUE= in the environment).
> > 
> > This isn't really limited to OS X. My understanding is that there's no
> > Unix or Unix-like system that supports non-blocking IO on regular
> > files except maybe through the special aio interface described in the
> > following link. That interface is often unimplemented, buggy, and/or
> > emulated in userspace with blocking IO done from hidden threads. And
> > it's not supported by libevent, perhaps for these reasons.
> 
> Nonblocking I/O has not much to do with it, you don't necessarily need
> nonblocking I/O if you have working poll(2) or select(2).
> 
> It is a limitation of OS X. Every other platform with kqueue(2) and all
> I am aware of with poll(2) support it on all file descriptors. OS X
> doesn't support it on anything other than sockets - so not on ttys, not
> on files, not anything except sockets.
> 
> This is annoying because it means you can't manage tty file descriptors
> or file descriptors linked to devices such as /dev/null (both of which
> are commonly used for eg stdout) with libevent unless you force it to
> use select.
> 
> Linux epoll doesn't support /dev/null either, but it does support ttys
> and IIRC files too.
> 
> > 
> > http://pubs.opengroup.org/onlinepubs/009695399/basedefs/aio.h.html
> > 
> > In particular, read() and write() on "regular files" will never return
> > EAGAIN, usually use "uninterruptible IO" (meaning even a SIGKILL will
> > not terminate the thread/process doing the IO if the device is hung;
> > you'll just see the dreaded state "D" in ps), and don't really work
> > with any of the notification interfaces used by libevent. They may be
> > reported as always available, simply cause EINVAL or some such error,
> > or do something more strange.
> > 
> > fwiw, if you browse around the aio docs I linked above, you'll see the
> > following rationale:
> > 
> > """
> > Rationale for New Interface
> > 
> > Non-blocking I/O does not satisfy the needs of either realtime or
> > high-performance computing models; these models require that a process
> > overlap program execution and I/O processing. Realtime applications
> > will often make use of direct I/O to or from the address space of the
> > process, or require synchronized (unbuffered) I/O; they also require
> > the ability to overlap this I/O with other computation. In addition,
> > asynchronous I/O allows an application to keep a device busy at all
> > times, possibly achieving greater throughput. Supercomputing and
> > database architectures will often have specialized hardware that can
> > provide true asynchrony underlying the logical asynchrony provided by
> > this interface. In addition, asynchronous I/O should be supported by
> > all types of files and devices in the same manner.
> > """
> > 
> > I don't think that's entirely accurate or an adequate justification
> > for not supporting the existing (and much simpler) non-blocking
> > interfaces, but I wasn't in the room when the decision was made...
> > 
> > libevent works well with sockets (including TCP, UDP, and Unix-domain)
> > and pipes (named and unnamed) but not so well with regular files.
> > 
> > >
> > >
> > > On Thu, Sep 01, 2011 at 11:19:52PM +0100, Bernd Schoeller wrote:
> > >> Dear List,
> > >>
> > >> I am currently taking my first steps in experimenting with libevent.
> > >> To do that, I have written the small example below that should copy
> > >> a file. I am currently trying to restrict myself to the core
> > >> functions, and I know that the buffer handling is too primitive.
> > >>
> > >> I am trying the code on 3 different platforms:
> > >>
> > >> a) On OpenBSD 4.9, the code seems to work and the file is copied. I
> > >> am not sure what version OpenBSD is using and what they changed.
> > >>
> > >> b) On MacOS Lion, using libevent 2.0.10, the program copies the
> > >> file, but seems to never receive the 0 bytes EOF read, thus blocking
> > >> after the last write. Output ends with:
> > >>
> > >> ...
> > >> read callback
> > >> Read: 6, 2 - 81953
> > >> write callback
> > >> Written: 7, 4 - 81953 (81953) of 81953
> > >> Sub read.
> > >>
> > >> c) On Linux (Ubuntu LTS), using libevent 1.4.13, the read operation
> > >> is never triggered and the main loop exits immediately. The output
> > >> printed is:
> > >>
> > >> Starting.
> > >> Opened read file handle 6.
> > >> Opened write file handle 7.
> > >> Exited with 1
> > >>
> > >> With these 3 different behaviors, I am pretty sure that I have not
> > >> understood some fundamental detail of libevent. I do not think the
> > >> different versions are to blame, and it should be possible to create
> > >> some code that runs on all versions, as I am using very basic
> > >> functions that seem to be the same in all versions.
> > >>
> > >> Thanks for your help,
> > >> Bernd
> > >>
> > >>
> > >> ---> c_libevent_test.c <---
> > >>
> > >> #include <event.h>
> > >> #include <stdio.h>
> > >> #include <fcntl.h>
> > >> #include <sys/types.h>
> > >> #include <sys/stat.h>
> > >>
> > >> #define READ_FILE_NAME "in.dat"
> > >> #define WRITE_FILE_NAME "out.dat"
> > >>
> > >> #define BUFFER_SIZE 100000
> > >>
> > >> struct event fd_read;
> > >> struct event fd_write;
> > >>
> > >> int buffer[BUFFER_SIZE];
> > >> int full;
> > >> int written;
> > >>
> > >> void onRead(int fd, short mode, void* data)
> > >> {
> > >> ? printf("read callback\n");
> > >> ? full = read(fd, buffer, BUFFER_SIZE);
> > >> ? printf("Read: %d, %d - %d\n", fd, mode, full);
> > >> ? if (full > 0) {
> > >> ? ? written = 0;
> > >> ? ? event_add(&fd_write, NULL);
> > >> ? }
> > >> }
> > >>
> > >> void onWrite(int fd, short mode, void *data)
> > >> {
> > >> ? printf("write callback\n");
> > >> ? int wr = write(fd, buffer+written, full-written);
> > >> ? written += wr;
> > >> ? printf("Written: %d, %d - %d (%d) of %d\n", fd, mode,
> > >> ? ? ? ? ?wr, written, full);
> > >> ? if (wr >= full) {
> > >> ? ? written = 0;
> > >> ? ? full = 0;
> > >> ? ? printf("Sub read.\n");
> > >> ? ? event_add(&fd_read, NULL);
> > >> ? } else {
> > >> ? ? event_add(&fd_write, NULL);
> > >> ? }
> > >> }
> > >>
> > >> int main(int argc, char** argv)
> > >> {
> > >> ? full = 0;
> > >>
> > >> ? event_init();
> > >>
> > >> ? printf("Starting.\n");
> > >>
> > >> ? int fd;
> > >> ? fd = open(READ_FILE_NAME, O_RDONLY);
> > >> ? printf("Opened read file handle %d.\n", fd);
> > >> ? event_set(&fd_read, fd, EV_READ, onRead, 0);
> > >>
> > >> ? fd = open(WRITE_FILE_NAME, O_WRONLY | O_CREAT, 0600);
> > >> ? printf("Opened write file handle %d.\n", fd);
> > >> ? event_set(&fd_write, fd, EV_WRITE, onWrite, 0);
> > >>
> > >> ? event_add(&fd_read, NULL);
> > >> ? printf("Exited with %d\n", event_dispatch());
> > >> ? return 0;
> > >> }
> > >>
> > >> ***********************************************************************
> > >> To unsubscribe, send an e-mail to majordomo@xxxxxxxxxxxxx with
> > >> unsubscribe libevent-users ? ?in the body.
> > > ***********************************************************************
> > > To unsubscribe, send an e-mail to majordomo@xxxxxxxxxxxxx with
> > > unsubscribe libevent-users ? ?in the body.
> > >
> > 
> > 
> > 
> > -- 
> > Scott Lamb <http://www.slamb.org/>
> > ***********************************************************************
> > To unsubscribe, send an e-mail to majordomo@xxxxxxxxxxxxx with
> > unsubscribe libevent-users    in the body.
> ***********************************************************************
> To unsubscribe, send an e-mail to majordomo@xxxxxxxxxxxxx with
> unsubscribe libevent-users    in the body.
***********************************************************************
To unsubscribe, send an e-mail to majordomo@xxxxxxxxxxxxx with
unsubscribe libevent-users    in the body.

References:
- [Libevent-users] Different behavior of my code
  - From: Bernd Schoeller
- Re: [Libevent-users] Different behavior of my code
  - From: Nicholas Marriott
- Re: [Libevent-users] Different behavior of my code
  - From: Scott Lamb
- Re: [Libevent-users] Different behavior of my code
  - From: Nicholas Marriott

Prev by Author: Re: [Libevent-users] Different behavior of my code
Next by Author: Re: [Libevent-users] Different behavior of my code
Previous by thread: Re: [Libevent-users] Different behavior of my code
Next by thread: Re: [Libevent-users] Different behavior of my code
Index(es):
- Author
- Thread