For HTTP connections, QNAM defaults to opening its TCP socket in
unbuffered mode. This means that Qt will send the data written
into the socket right to the kernel, queueing only if the kernel
says it doesn't want more data for the moment being.
QNAM itself then uses separate write() calls to write the HTTP
headers and the body of the request (like POST or PUT). These 2+
writes result in headers and body being sent over different TCP
segments -- even if, in principle, a POST with a few bytes of data
(e.g. a HTML form, or a REST or SOAP request) could fit in the same
segment as the request.
Multiple writes like this interact extremely poorly with other
TCP features, e.g. delayed ACKs, Nagle's algorithm and the like.
In a typical scenario, the kernel will send a segment containing just
the headers, wait for the ACK (which may be delayed), and only then
send the body (it wasn't sent before because Nagle was blocking it).
The reply at this point is immediate (because the server can process
the request and starts replying), but the delayed ACK is typically
40-50ms, and documented up to 500ms (!). If one uses QNAM to access a
service, this introduces unacceptable latency.
These multiple writes to the OS should be avoided.
The first thing that comes into mind is to use buffered sockets.
Now, there are good reasons to keep the socket unbuffered, so we
don't want to change that. But the deal breaker is that even buffered
sockets won't help in general: for instance, on Windows, a buffered
write will immediately detect that the socket is ready for write and
flush the buffer right away (not 100% sure of why this is necessary;
basically, after populating the QTcpSocket write buffer, Qt enables
a write socket notifier on the socket -- notifier that fires
synchronously and immediately, without even returning to the event
loop, and that causes the write buffer flush).
Linux of course offers the perfect solution: corking the socket via
TCP_CORK, which tells the kernel not to send the data right away but
to buffer it up to a timeout (or when the option gets disabled
again, whichever comes first). It's explicitly designed to support
the case of sending headers followed by something like a
sendfile(2). Setting this socket option moves the problem to
the kernel and we could happily keep issuing multiple writes.
Ça va sans dire, no other OS supports that option or any other
similar option.
We have therefore to deal with this in userspace: don't write in the
socket multiple times, but try and coalesce the write of the headers
with the writing of the data. This patch implements that, by storing
the headers and sending them together with the very first chunk of
data. If the data is small enough, this sends the entire request
in one TCP segment.
Interestingly enough, QNAM has a call setting TCP_NODELAY
currently commented out because Qt doesn't combine "HTTP requests"
(whatever that means). The call comes all the way back
from pre-public history (before 2011) (!). This patch doesn't
touch it.
Fixes: QTBUG-41907
Change-Id: Id555d14e0702c9f75c3134b18277692eb3659afe
Reviewed-by: Mårten Nordheim <marten.nordheim@qt.io>
7a5e0c5712