Re: [Libevent-users] Cross-linked socket and openssl bufferevent throughput issue

[Azat Khuzhin <azat@xxxxxxxxxxxx>]

Hi!

> This appears to work however in testing the throughput of the overall system is very low

And "very low" is?
What is the maximum throughput of the bufferevent in your case?

> When examining the running process with strace I noticed that each event
> loop itteration is limited to a 4096 byte read in spite of running with a
> much larger setting in the bufferevent (I used the bufferevent_set_max*
> functions to set maximums for read and write).  The 4096 limit appears to be
> hard-coded in buffer.c, is that correct?

Indeed, I'm aware of it for around half a year.

The reason that it is not fixed yet is that:
- personally I wanted to see how the memory fragmentation will work
after increasing this buffer and provide some numbers in the patch
- I was waiting the patch from a guy that initially came across this

And also keep in mind that the problem in your case can be in openssl, due to:
- CPU usage
- and that fact that openssl code sometimes reads lower chunks than it
could (I have some examples, but they are only in mailbox)

Plus I just did some testing and here what I got with simple echo server:
- plain bufferevent: 1GB/s
- openssl bufferevent: ~400MB/s

(on intel i7 8550u)

Also note that this 4k limit *may not affect* openssl bufferevent, since:
- openssl bufferevent that created with underlying bufferevent is
still affected (due to it uses buffer of the bufferevent via BIO
wrappers)
- you need to set high watermark for bufferevent to overcome 4k limit
there (doh)
- for more details see bytes_to_read() in bufferevent_openssl.c

> If so is there any way to increase
> the limit as it appears that with this limit the system is unable to
> keep up with the amount of traffic (the strace output shows increasing
> amounts of data available for reading).

Yep it is hard coded in buffer.c

> If not are there any other settings
> I can alter in libevent to tune it for this use-case?

You can add an API for evbuffer to change the default size and create
a pull request with your changes here [2]. Are you interested in this?
Or should I?

  [2]: https://github.com/libevent/libevent
***********************************************************************
To unsubscribe, send an e-mail to majordomo@xxxxxxxxxxxxx with
unsubscribe libevent-users    in the body.