Happy new year folks,
I am trying to chase down an issue which maybe be related to bufferevent. I am using bufferevent library to manage my TCP sessions. For TCP writes, instead bufferevent_write(), I use evbuffer_add_reference() to avoid copy of the data. TCP sessions exchange HELLO packets in addition to other relevant data. I am seeing a peculiar behavior.
A -------- B
B is sending lots of data to A, while A is just sending Hello packets to B. When A is sending these hello packets, I see a huge latency (in order of seconds) on B. Through my internal debugs, I see that A is sending the hello packets when it is suppose to i.e. I see the debug on A that it called 'evbuffer_add_reference', but on B I don't see it. B periodically does check if there is any data from A as well.
Q1) Does evbuffer_add_reference() call write() syscall synchronously or does it try to accumulate more data and then calls write()?
If evbuffer_add_reference() does not call write() in its call path, is there a way to force it? I see there is bufferevent_flush() API. Could this be used to force a write?
Although, be_socket_flush() does not do anything.
static int
be_socket_flush(struct bufferevent *bev, short iotype,
enum bufferevent_flush_mode mode)
{
return 0;
}
It seems that when A is calling evbuffer_add_reference(), the data is not sent immediately. I do have TCP_NODELAY option set on the socket.
Regards,
-Himanshu