Re: [Libevent-users] Design struggle

[Azat Khuzhin <a3at.mail@xxxxxxxxx>]

On Thu, Sep 04, 2014 at 10:43:55AM +0200, B.R. wrote:
> On Thu, Sep 4, 2014 at 9:40 AM, Azat Khuzhin <a3at.mail@xxxxxxxxx> wrote:
> 
> > On Fri, Aug 29, 2014 at 05:53:50PM +0200, B.R. wrote:
> > > 1Â) Filters are created cointaining information of connections they
> > should
> > > be bound to (protocol, port), events that should trigger them and a
> >
> > I guess only one of this: protocol || port
> >
> 
> âThat was just a description, not a question actually.â
> 
> âFilters work on a specific protocol+port,â I require them to provide both
> pieces of information.
> This allows me to:
> - Automatically determine which port is to be listened to for which
> protocol (auto-determination of sockets to create/bind/listen to)
> - Have multiple filters on the same tuple (protocol, port), sharing the
> read information and writing to the socket if an answer they can provide is
> needed
> 
> > I) If multiple filters ask to be triggered on READ event on the same
> > > socket, what will they see? Will they all be able to read the same
> > message
> > > which triggered the event and not another one (ie another message
> > received
> > > while processing the filter list)?
> >
> > It depends from how you use this.
> >
> > Kernel will just return array of bytes when you call read() on some
> > fd/socket, and once you read it, the second read() will never return the
> > same portion of data.
> >
> > You could use evbuffer, to save data, and then feed it to every filter
> > that assigned to this fd/socket, and drain it when you finished.
> >
> 
> âI cannot use evbuffer, though it was the initial (and the best) idea.
> Remember I support bot TCP and UDP.
> Do evbuffers now handle UDP? The docs and user reports still indicate the
> contrary.â

Nope, evbuffers can't work with UDP.

> 
> 
> >
> > > II) If multiple filters wanna write data to the socket, is it safe to
> > have
> > > each filter having its separate buffer and triggering its own WRITE event
> > > on it?
> > ââ
> >
> > This is not safe, because write(2) not always will write all you portion
> > of data, it could write less then @count, and return this value to
> > libevent, and then event loop will schedule the second WRITE event for
> > the same fd/socket:
> >
> > write(10, foobar[4096]foobaz[4096], 8192) = 4096 # event1
> > write(10, barfoo[4096]bazfoo[4096], 8192) = 4096 # event2
> > write(10, foobaz[4096], 4096) = 4096 # event1
> > write(10, bazfoo[4096], 4096) = 4096 # event2
> > ...
> > read(10, buf, 8192*2) = 8192*2
> >
> > Actual:
> > buf = foobar[4096]barfoo[4096]foobaz[4096]bazfoo[4096]
> > Expected:
> > buf = foobar[4096]foobaz[4096]barfoo[4096]bazfoo[4096]
> >
> 
> âActually, I moved on from than and, even if I am using separate buffers
> for incoming data, I share the write event (bound to the fd) among all the
> filters.
> When writing to the buffer, I take caution not to overwrite existing data
> and use the buffer content counter to append data if some is already
> existing.
> 
> But yeah I have a problem: I cannot know which filter wrote which part of
> the buffer...
> And I cannot multiplex writings to the output buffer since, as you pointed
> out, there is no way of knowing how the asynchronous system will dispatch
> the WRITE events to the socket.
> 
> Actually, it looks to me as a false problem:
> - In UDP, there is no 'stream' concept, only single isolated datagrams.
> That is simple: for each packet, you decide to write something or not
> depending on its processing. There is no 'chunk' concept, and you write
> everything in one shot: there is no multiplexing, hence no other filter
> might interleave data.
> - In TCP, you deal with a stream, and there is no way of knowing if sent
> data will be received in any number of packets. That is why you check
> bounds and if not all the data has been received, you then wait for more to
> come. The mistake here would be to partially process data and if possible
> send a partial answer: interleaving might happen from other filters, making
> the sent message gibberish.
> 
> Partial handling is a nice feature of stream sockets, however the
> asynchronous nature of their processing in our case means there is no
> control over the filters multiplexing. The other solution would be to
> attempt to totally order handlers to avoid interleaving, but that woudl
> require a global state, locking... and rendering this magnificent
> asynchronous system synchronous again... great. ^^
> 
> Since data is now appended to the same buffer, associated in a single event
> to the same socket fd, I suppose everything is totally ordered now. Am I
> right?

Right (at least for TCP).

> Do you notice any flaw there?
> 
> 
> >  >
> > > Here is a use case summing up both previous inquiries:
> > > Say a filter wrote data to a socket after parsing the content it read on
> > > it, and that the peer reacted to that answer, what will subsequent
> > filters,
> > > for which the READ event has been triggered by the initial message, read?
> > > a) Initial message?
> > > b) Either initial message or answer (race, undecided)?
> > > c) Nothing since the event has been canceled (not pending anymore), the
> > > subsequent filters will only receive a new event for READ on reception of
> > > the answer
> > >
> > > I am thinking of using a dispatcher which would sequentially (and
> > manually)
> > > trigger the event of each of the filters. However that implies not
> > linking
> >
> > Why you want to do it manually?
> >
> 
> âBecause binding several filters to the same fd means that only one of
> themâ will receive information.
> Multiple recv/recvfrom on the same socket returns the received information
> for only one call, the others being empty.
> 
> I then recv/recvfrom once and populate individual filters buffer with it
> (by appending data). Now each filter can handle it safely and play with
> their own buffer content counter to indicate where new data shall be
> inserted by the data-receiving event handler.

Yep, now I understand, just like I said above.

>  â
> 
> > > the filters event with the socket, thus forcing me to maintain a separate
> > > buffer for each of the filter (with memory and processing overhead that
> > > implies). Moreover, the problem reappears if another message is received
> > on
> > > the socket while the dispatching loop is at work... and the sequential
> > work
> > > makes the benefit of the event-based system disappear!
> >
> > Which problem?
> > You mean that you filter-handling stuff works significantly slower than
> > read/write?
> >
> > â
> The problem of not being able to read the same information from the socket
> several times (which is normal).
> 
> Another part of the problem is what happens if one of the filters might be
> at work while several messages are received on the wire. I do not want any
> filter to miss any part of any received message.â
> 
> â I thought that, by handling the push of the received information to the
> filters buffer individually, after having checked that their own event (the
> one associated with their callback procedure)â
> ââ is not active (running), I would prevent that loss.â
> 
> Here is a sample case:
> received "message1"
> triggers READ event on handler1 -> handler1 reads "message1" and processes
> it
> triggers READ event on handler2 -> handler2 reads "message2" and processes
> it

s/message2/message1/ I guess.

> 
> received "message2"
> triggers READ event on handler1 (still busy processing)
> triggers READ event on handler2 -> handler2 reads "message2" and processes
> it
> 
> received "message3"
> triggers READ event on handler1 -> handler1 reads "message3" and processes
> it
> triggers READ event on handler2 -> handler2 reads "message3" and processes
> it
> 
> Whoops! handler1 missed "message2"...

Why it will missed "message2", if you filled handler1.buffer with
"message2" and then "message3", it must handle firstly "message2" and
only after this "message3".
Am I missing something?

> 
> >
> > > I do not know if the bufferevents would be useful to my case. Anyway,
> > those
> > > are not an option, since part of the traffic I am handling is UDP
> > datagrams.
> >
> 
> âI said I was using UDP too. evbuffers seem to be of no help to me...
> ---
> *B. R.*

-- 
Respectfully
Azat Khuzhin
***********************************************************************
To unsubscribe, send an e-mail to majordomo@xxxxxxxxxxxxx with
unsubscribe libevent-users    in the body.