Flushing large data immediately in pqcomm

Hi hackers

I've been looking into ways to reduce the overhead we're having in pqcomm and I'd like to propose a small patch to modify how socket_putmessage works.

Currently socket_putmessage copies any input data into the pqcomm send buffer (PqSendBuffer) and the size of this buffer is 8K. When the send buffer gets full, it's flushed and we continue to copy more data into the send buffer until we have no data left to be sent.

Since the send buffer is flushed whenever it's full, I think we are safe to say that if the size of input data is larger than the buffer size, which is 8K, then the buffer will be flushed at least once (or more, depends on the input size) to store and all the input data.

Proposed change modifies socket_putmessage to send any data larger than 8K immediately without copying it into the send buffer. Assuming that the send buffer would be flushed anyway due to reaching its limit, the patch just gets rid of the copy part which seems unnecessary and sends data without waiting.

This change affects places where pq_putmessage is used such as pg_basebackup, COPY TO, walsender etc.

I did some experiments to see how the patch performs. 

Firstly, I loaded ~5GB data into a table [1], then ran "COPY test TO STDOUT". Here are perf results of both the patch and HEAD

HEAD:

The patch brings a ~5% gain in socket_putmessage.

Also timed the pg_basebackup like: 

time pg_basebackup -p 5432 -U replica_user -X none -c fast --no_maanifest -D test

HEAD:

Patch:

It seems ~11% faster in this specific case.

I'd appreciate any feedback/thoughts.

[1]

CREATE TABLE test(id int, name text, time TIMESTAMP);