Hi all,
As was previously discussed at the thread surrounding [1]: Currently
any block registration in WAL takes up at least 8 bytes in xlog
overhead, regardless of how much data is included in that block: 1
byte for block ID, 1 byte for fork and flags, 2 bytes for the block
data length, and 4 bytes for the blockNo. (Usually, another 12 bytes
are used for a RelFileLocator; but that may not be included for some
blocks in the record when conditions apply)
Attached is a patch that reduces this overhead by up to 2 bytes by
encoding how large the block data length field is into the block ID,
and thus optionally reducing the block data's length field to 0 bytes.
Examples: cross-page update records will now be 2 bytes shorter,
because the record never registers any data for the new block of the
update; pgbench transactions are now either 6 or 8 bytes smaller
depending on whether the update crosses a page boundary (in xlog
record size; after alignment it is 0 or 4/8 bytes, depending on
MAXALIGN and whether the updates are cross-page updates).
It changes the block IDs used to fit in 6 bits, using the upper 2 bits
of the block_id field to store how much data is contained in the
record (0, <=UINT8_MAX, or <=UINT16_MAX bytes).
This is part 1 of a series of patches trying to decrease the size of
WAL; see also [0], [1] and [2] for more info on what's still to go.
I'm working on a separate, much more involved patch for the XLogRecord
header itself, similar to the patch in [1], which I expect to send
sometime soon as well.
Unless someone thinks the patches should be discussed as one series,
I'm planning on posting that in another thread, as I don't see any
meaningful dependencies between the patches, and the XLR header patch
will be quite a bit larger than this one.
Kind regards,
Matthias van de Meent
Neon, Inc.
[0] https://wiki.postgresql.org/wiki/Updating_the_WAL_infrastructure
[1]
https://www.postgresql.org/message-id/flat/CAEze2Wjd3jY_UhhOGdGGnC6NO%3D%2BNmtNOmd%3DJaYv-v-nwBAiXXA%40mail.gmail.com#17a51d83923f4390d8f407d0d6c5da07
[2]
https://www.postgresql.org/message-id/flat/CAEze2Whf%3DfwAj7rosf6aDM9t%2B7MU1w-bJn28HFWYGkz%2Bics-hg%40mail.gmail.com
PS. Benchmark results on my system (5950x with other light tasks
running) don't show an obviously negative effect in a 10-minute run
with these arbitrary pgbench settings on a fresh cluster with default
configuration:
./pg_install/bin/pgbench postgres -j 2 -c 6 -T 600 -M prepared
[...]
master: tps = 375
patched: tps = 381