Re: [Libevent-users] [Libevent 2.0.x] Broadcast TCP chunks of data using libevent

[Nick Mathewson <nickm@xxxxxxxxxxxxx>]

On Mon, Oct 12, 2009 at 07:41:22PM -0700, Raine Fan wrote:
[...]

> Let's suppose my app has to serve about 1K TCP connections. Since
> the app has, let's say, 10 different feeds to broadcast to different
> groups of connections (which any group can have any connection id,
> even overlapped connection id's on different groups; where each
> group may serve a unique feed).

Your question confused me at first, since my first reaction was to
think you meant 'broadcast' in the sense of IP unicast/multicast/
broadcast, and say, "But TCP doesn't support broadcast addresses!"

Instead, I _think_ you mean "send the same content on a large number
of TCP connections".

> 1) Does libevent 2.0 has the possibility to broadcast (in TCP terms)
> a chunk of data to all groups of connection ids with only
> _one_single_ libevent system call (let's say a zero-copy like) ?

I'm not quite sure what mean by a "libevent system call".  I'm going
to assume that what you want to do is minimize the number of data
copies and the number of system calls made to the kernel by libevent.
Calls _to_ libevent are not system calls, since libevent isn't part of
the kernel.

I'll assume you're using bufferevents, since you're talking about TCP
abstractions in Libevent 2.0.

The short answer is "Not in the way you're asking for."  There will be
at least one system call from Libevent to the kernel per fd that
you're writing to.  I don't know of a kernel interface that we could
wrap in order to get fewer than one call per fd myself.  (If anybody
knows of some fancy "write this data to the following array of fds"
syscall out there, please let me know.)

That said, here are some ways you can minimize data copying:

If your content is in a file, you can use the evbuffer_add_file()
function to use your system's mmap/sendfile/splice capabilities as
warranted.  In this case, no data should need to be copied from
user memory to kernel memory, and it's up to the kernel to decide how
much copying it wants to do in its network stack.

This interface isn't optimized for sending many copies or extents from
a single file, but it shouldn't be too bad for that; it would be nice
to have one that was.

If your content is bytes in RAM that you don't want to copy, you'd use
the evbuffer_add_reference() interface to make sure that Libevent
doesn't make any internal copies of your data before passing it to the
kernel.  You need to provide a cleanup function when using this
interface, so that Libevent can tell your application once it is no
longer using the memory.

> If not, is there any abstract API in libevent core that I could
> implement my broadcast behaviour?

You'd probably want to have a look at how bufferevents and evbuffers
look now, and see what you want to do with them.

> 2) In postive case, would i have to use different event_base loop
> for each group of connection ids (each group is a feed of multimedia
> data) or libevent would allow me to create different groups inside a
> same event_base loop?

I see no reason you'd need multiple event bases for this.

> 3) Would any one help me with a example or a tip?
>
>  4) In the negative case, is any possibility to have this
> implemented in future realeases? It really would be a great idea
> since this would the overhead of the same system call to each of
> connection id's in any group!

Like I said above, I don't know of any system call provided by any
kernel that supports writing to large numbers of fds at once in a way
that would be useful to you---but I'm no expert on kernel esoterica,
and there might well be one that I haven't heard of.  If some OS
supports this, I agree that it would be swell to have Libevent able to
take advantage.

peace,
-- 
Nick
***********************************************************************
To unsubscribe, send an e-mail to majordomo@xxxxxxxxxxxxx with
unsubscribe libevent-users    in the body.