[Author Prev][Author Next][Thread Prev][Thread Next][Author Index][Thread Index]

Re: [Libevent-users] Different behavior of my code



It is however true to say that files will always return as readable and
writable but that still means kqueue and poll and hence libevent works
:-).


On Thu, Sep 08, 2011 at 04:44:26PM +0100, Nicholas Marriott wrote:
> On Thu, Sep 08, 2011 at 01:17:22AM -0700, Scott Lamb wrote:
> > On Tue, Sep 6, 2011 at 3:25 AM, Nicholas Marriott
> > <nicholas.marriott@xxxxxxxxx> wrote:
> > > OpenBSD is libevent 1.4.13.
> > >
> > > OS X polling mechanisms are pretty bad: their poll() and kqueue() only
> > > support sockets, so you will need to use select (set EVENT_NOPOLL=1
> > > EVENT_NOKQUEUE= in the environment).
> > 
> > This isn't really limited to OS X. My understanding is that there's no
> > Unix or Unix-like system that supports non-blocking IO on regular
> > files except maybe through the special aio interface described in the
> > following link. That interface is often unimplemented, buggy, and/or
> > emulated in userspace with blocking IO done from hidden threads. And
> > it's not supported by libevent, perhaps for these reasons.
> 
> Nonblocking I/O has not much to do with it, you don't necessarily need
> nonblocking I/O if you have working poll(2) or select(2).
> 
> It is a limitation of OS X. Every other platform with kqueue(2) and all
> I am aware of with poll(2) support it on all file descriptors. OS X
> doesn't support it on anything other than sockets - so not on ttys, not
> on files, not anything except sockets.
> 
> This is annoying because it means you can't manage tty file descriptors
> or file descriptors linked to devices such as /dev/null (both of which
> are commonly used for eg stdout) with libevent unless you force it to
> use select.
> 
> Linux epoll doesn't support /dev/null either, but it does support ttys
> and IIRC files too.
> 
> > 
> > http://pubs.opengroup.org/onlinepubs/009695399/basedefs/aio.h.html
> > 
> > In particular, read() and write() on "regular files" will never return
> > EAGAIN, usually use "uninterruptible IO" (meaning even a SIGKILL will
> > not terminate the thread/process doing the IO if the device is hung;
> > you'll just see the dreaded state "D" in ps), and don't really work
> > with any of the notification interfaces used by libevent. They may be
> > reported as always available, simply cause EINVAL or some such error,
> > or do something more strange.
> > 
> > fwiw, if you browse around the aio docs I linked above, you'll see the
> > following rationale:
> > 
> > """
> > Rationale for New Interface
> > 
> > Non-blocking I/O does not satisfy the needs of either realtime or
> > high-performance computing models; these models require that a process
> > overlap program execution and I/O processing. Realtime applications
> > will often make use of direct I/O to or from the address space of the
> > process, or require synchronized (unbuffered) I/O; they also require
> > the ability to overlap this I/O with other computation. In addition,
> > asynchronous I/O allows an application to keep a device busy at all
> > times, possibly achieving greater throughput. Supercomputing and
> > database architectures will often have specialized hardware that can
> > provide true asynchrony underlying the logical asynchrony provided by
> > this interface. In addition, asynchronous I/O should be supported by
> > all types of files and devices in the same manner.
> > """
> > 
> > I don't think that's entirely accurate or an adequate justification
> > for not supporting the existing (and much simpler) non-blocking
> > interfaces, but I wasn't in the room when the decision was made...
> > 
> > libevent works well with sockets (including TCP, UDP, and Unix-domain)
> > and pipes (named and unnamed) but not so well with regular files.
> > 
> > >
> > >
> > > On Thu, Sep 01, 2011 at 11:19:52PM +0100, Bernd Schoeller wrote:
> > >> Dear List,
> > >>
> > >> I am currently taking my first steps in experimenting with libevent.
> > >> To do that, I have written the small example below that should copy
> > >> a file. I am currently trying to restrict myself to the core
> > >> functions, and I know that the buffer handling is too primitive.
> > >>
> > >> I am trying the code on 3 different platforms:
> > >>
> > >> a) On OpenBSD 4.9, the code seems to work and the file is copied. I
> > >> am not sure what version OpenBSD is using and what they changed.
> > >>
> > >> b) On MacOS Lion, using libevent 2.0.10, the program copies the
> > >> file, but seems to never receive the 0 bytes EOF read, thus blocking
> > >> after the last write. Output ends with:
> > >>
> > >> ...
> > >> read callback
> > >> Read: 6, 2 - 81953
> > >> write callback
> > >> Written: 7, 4 - 81953 (81953) of 81953
> > >> Sub read.
> > >>
> > >> c) On Linux (Ubuntu LTS), using libevent 1.4.13, the read operation
> > >> is never triggered and the main loop exits immediately. The output
> > >> printed is:
> > >>
> > >> Starting.
> > >> Opened read file handle 6.
> > >> Opened write file handle 7.
> > >> Exited with 1
> > >>
> > >> With these 3 different behaviors, I am pretty sure that I have not
> > >> understood some fundamental detail of libevent. I do not think the
> > >> different versions are to blame, and it should be possible to create
> > >> some code that runs on all versions, as I am using very basic
> > >> functions that seem to be the same in all versions.
> > >>
> > >> Thanks for your help,
> > >> Bernd
> > >>
> > >>
> > >> ---> c_libevent_test.c <---
> > >>
> > >> #include <event.h>
> > >> #include <stdio.h>
> > >> #include <fcntl.h>
> > >> #include <sys/types.h>
> > >> #include <sys/stat.h>
> > >>
> > >> #define READ_FILE_NAME "in.dat"
> > >> #define WRITE_FILE_NAME "out.dat"
> > >>
> > >> #define BUFFER_SIZE 100000
> > >>
> > >> struct event fd_read;
> > >> struct event fd_write;
> > >>
> > >> int buffer[BUFFER_SIZE];
> > >> int full;
> > >> int written;
> > >>
> > >> void onRead(int fd, short mode, void* data)
> > >> {
> > >> ? printf("read callback\n");
> > >> ? full = read(fd, buffer, BUFFER_SIZE);
> > >> ? printf("Read: %d, %d - %d\n", fd, mode, full);
> > >> ? if (full > 0) {
> > >> ? ? written = 0;
> > >> ? ? event_add(&fd_write, NULL);
> > >> ? }
> > >> }
> > >>
> > >> void onWrite(int fd, short mode, void *data)
> > >> {
> > >> ? printf("write callback\n");
> > >> ? int wr = write(fd, buffer+written, full-written);
> > >> ? written += wr;
> > >> ? printf("Written: %d, %d - %d (%d) of %d\n", fd, mode,
> > >> ? ? ? ? ?wr, written, full);
> > >> ? if (wr >= full) {
> > >> ? ? written = 0;
> > >> ? ? full = 0;
> > >> ? ? printf("Sub read.\n");
> > >> ? ? event_add(&fd_read, NULL);
> > >> ? } else {
> > >> ? ? event_add(&fd_write, NULL);
> > >> ? }
> > >> }
> > >>
> > >> int main(int argc, char** argv)
> > >> {
> > >> ? full = 0;
> > >>
> > >> ? event_init();
> > >>
> > >> ? printf("Starting.\n");
> > >>
> > >> ? int fd;
> > >> ? fd = open(READ_FILE_NAME, O_RDONLY);
> > >> ? printf("Opened read file handle %d.\n", fd);
> > >> ? event_set(&fd_read, fd, EV_READ, onRead, 0);
> > >>
> > >> ? fd = open(WRITE_FILE_NAME, O_WRONLY | O_CREAT, 0600);
> > >> ? printf("Opened write file handle %d.\n", fd);
> > >> ? event_set(&fd_write, fd, EV_WRITE, onWrite, 0);
> > >>
> > >> ? event_add(&fd_read, NULL);
> > >> ? printf("Exited with %d\n", event_dispatch());
> > >> ? return 0;
> > >> }
> > >>
> > >> ***********************************************************************
> > >> To unsubscribe, send an e-mail to majordomo@xxxxxxxxxxxxx with
> > >> unsubscribe libevent-users ? ?in the body.
> > > ***********************************************************************
> > > To unsubscribe, send an e-mail to majordomo@xxxxxxxxxxxxx with
> > > unsubscribe libevent-users ? ?in the body.
> > >
> > 
> > 
> > 
> > -- 
> > Scott Lamb <http://www.slamb.org/>
> > ***********************************************************************
> > To unsubscribe, send an e-mail to majordomo@xxxxxxxxxxxxx with
> > unsubscribe libevent-users    in the body.
> ***********************************************************************
> To unsubscribe, send an e-mail to majordomo@xxxxxxxxxxxxx with
> unsubscribe libevent-users    in the body.
***********************************************************************
To unsubscribe, send an e-mail to majordomo@xxxxxxxxxxxxx with
unsubscribe libevent-users    in the body.