Re: XLog size reductions: smaller XLRec block header for PG17
От | Heikki Linnakangas |
---|---|
Тема | Re: XLog size reductions: smaller XLRec block header for PG17 |
Дата | |
Msg-id | 67c1ae87-fe76-09d8-a588-cf9bb2334b33@iki.fi обсуждение исходный текст |
Ответ на | XLog size reductions: smaller XLRec block header for PG17 (Matthias van de Meent <boekewurm+postgres@gmail.com>) |
Ответы |
Re: XLog size reductions: smaller XLRec block header for PG17
(Matthias van de Meent <boekewurm+postgres@gmail.com>)
Re: XLog size reductions: smaller XLRec block header for PG17 (Andres Freund <andres@anarazel.de>) |
Список | pgsql-hackers |
On 18/05/2023 17:59, Matthias van de Meent wrote: > Attached is a patch that reduces this overhead by up to 2 bytes by > encoding how large the block data length field is into the block ID, > and thus optionally reducing the block data's length field to 0 bytes. > Examples: cross-page update records will now be 2 bytes shorter, > because the record never registers any data for the new block of the > update; pgbench transactions are now either 6 or 8 bytes smaller > depending on whether the update crosses a page boundary (in xlog > record size; after alignment it is 0 or 4/8 bytes, depending on > MAXALIGN and whether the updates are cross-page updates). > > It changes the block IDs used to fit in 6 bits, using the upper 2 bits > of the block_id field to store how much data is contained in the > record (0, <=UINT8_MAX, or <=UINT16_MAX bytes). Perhaps we should introduce a few generic inline functions to do varint encoding. That could be useful in many places, while this scheme is very tailored for XLogRecordBlockHeader. We could replace XLogRecordDataHeaderShort and XLogRecordDataHeaderLong with this too. With just one XLogRecordDataHeader, with a variable-length length field. > This is part 1 of a series of patches trying to decrease the size of > WAL; see also [0], [1] and [2] for more info on what's still to go. > I'm working on a separate, much more involved patch for the XLogRecord > header itself, similar to the patch in [1], which I expect to send > sometime soon as well. > Unless someone thinks the patches should be discussed as one series, > I'm planning on posting that in another thread, as I don't see any > meaningful dependencies between the patches, and the XLR header patch > will be quite a bit larger than this one. > > Kind regards, > > Matthias van de Meent > Neon, Inc. > > [0] https://wiki.postgresql.org/wiki/Updating_the_WAL_infrastructure Good ideas here. Eliminating the two padding bytes from XLogRecord in particular seems like a pure win. > PS. Benchmark results on my system (5950x with other light tasks > running) don't show an obviously negative effect in a 10-minute run > with these arbitrary pgbench settings on a fresh cluster with default > configuration: > > ./pg_install/bin/pgbench postgres -j 2 -c 6 -T 600 -M prepared > [...] > master: tps = 375 > patched: tps = 381 That was probably not CPU limited, so that any overhead in generating the WAL would not show up. Try PGOPTIONS="-csynchronous_commit=off" and pgbench -N option. And make sure the scale is large enough that there is no lock contention. Also would be good to measure the overhead in replaying the WAL. How much space saving does this yield? - Heikki
В списке pgsql-hackers по дате отправления:
Следующее
От: Robert HaasДата:
Сообщение: Re: Introduce WAIT_EVENT_EXTENSION and WAIT_EVENT_BUFFER_PIN