Re: RFI: Extending the TOAST Pointer

Поиск
Список
Период
Сортировка
От Nikita Malakhov
Тема Re: RFI: Extending the TOAST Pointer
Дата
Msg-id CAN-LCVMsUb4vcoeJoPi=69LGym0Y2wmQYsduw+no097DT_T6gA@mail.gmail.com
обсуждение исходный текст
Ответ на Re: RFI: Extending the TOAST Pointer  (Aleksander Alekseev <aleksander@timescale.com>)
Ответы Re: RFI: Extending the TOAST Pointer  (Matthias van de Meent <boekewurm+postgres@gmail.com>)
Список pgsql-hackers
Hi,

Aleksander, I'm interested in extending TOAST pointer in various ways.
64-bit TOAST value ID allows to resolve very complex issue for production
systems with large tables and heavy update rate.

I agree with Matthias that there should not be processing of TOAST pointer
internals outside TOAST macros. Currently, TOASTed value is distinguished
as VARATT_IS_EXTERNAL_ONDISK, and it should stay this way. Adding
compression requires another implementation (extension) of
VARATT_EXTERNAL because current supports only 2 compression methods -
it has only 1 bit responsible for compression method, and there is a safe
way to do so, without affecting default TOAST mechanics - we must keep
it this way for compatibility issues and not to break DB upgrade.

Also, I must remind that we should not forget about field alignment inside
the TOAST pointer.

As it was already mentioned, it seems not very reasonable trying to save
a byte or two while we are storing out-of-line values of at least 2 kb in size.

On Mon, May 22, 2023 at 4:47 PM Aleksander Alekseev <aleksander@timescale.com> wrote:
Hi,

> I see your point, but I do think we should also think about why we do
> the change.

Personally at the moment I care only about implementing compression
dictionaries on top of this, as is discussed in the corresponding
thread [1]. I'm going to need new fields in the TOAST pointer's
including (but not necessarily limited to) dictionary id.

As I understand, Nikita is interested in implementing 64-bit TOAST
pointers [2]. I must admit I didn't follow that thread too closely but
I can imagine the needs are similar.

Last but not least I remember somebody on the mailing list suggested
adding ZSTD compression support for TOAST, besides LZ4. Assuming I
didn't dream it, the proposal was rejected due to the limited amount
of free bits in ToastCompressionId. It was argued that two possible
combinations that are left should be treated with care and ZSTD will
not bring enough value to the users compared to LZ4.

These are 3 recent cases I could recall. This being said I think our
solution should be generic enough to cover possible future cases
and/or cases unknown to us yet.

[1]: https://postgr.es/m/CAJ7c6TM7%2BsTvwREeL74Y5U91%2B5ymNobRbOmnDRfdTonq9trZyQ%40mail.gmail.com
[2]: https://commitfest.postgresql.org/43/4296/

--
Best regards,
Aleksander Alekseev


--
Regards,
Nikita Malakhov
Postgres Professional
The Russian Postgres Company

В списке pgsql-hackers по дате отправления:

Предыдущее
От: "Daniel Verite"
Дата:
Сообщение: Re: Should CSV parsing be stricter about mid-field quotes?
Следующее
От: Peter Geoghegan
Дата:
Сообщение: Re: Avoiding another needless ERROR during nbtree page deletion