Re: [PATCH] Infinite loop while acquiring new TOAST Oid

Поиск

Список

Период

Сортировка

От	Nikita Malakhov
Тема	Re: [PATCH] Infinite loop while acquiring new TOAST Oid
Дата	27 апреля 2023 г. 13:39:05
Msg-id	CAN-LCVP7Se50Ya0fC3q0ATJmO_o=JUdLMTSiwRaKWwc_2FOKHQ@mail.gmail.com обсуждение исходный текст
Ответ на	Re: [PATCH] Infinite loop while acquiring new TOAST Oid (Aleksander Alekseev <aleksander@timescale.com>)
Ответы	Re: [PATCH] Infinite loop while acquiring new TOAST Oid (Aleksander Alekseev <aleksander@timescale.com>)
Список	pgsql-hackers

Дерево обсуждения

Hi!

Widening of a TOAST pointer is possible if we keep backward compatibility with

old-fashioned TOAST tables - I mean differ 'long' and 'short' TOAST pointers and

process them accordingly on insert and delete cases, and vacuum with logical

replication. It is not very difficult, however it takes some effort. Recently I've found

out that I have not overseen all compatibility cases, so the provided patch is

functional but limited in compatibility.

We already have a flag byte in the TOAST pointer which is responsible for the type

of the pointer - va_flag field. It was explained in the Pluggable TOAST patch.

One is enough, there is no need to add another one.

On Wed, Apr 26, 2023 at 3:55 PM Aleksander Alekseev <aleksander@timescale.com> wrote:

Hi,

> I agree that we can't simply widen varatt_external to use 8 bytes for
> the toast ID in all cases.

+1

Note that the user may have a table with multiple TOASTable
attributes. If we simply widen the TOAST pointer it may break the
existing tables in the edge case. Also this may be a reason why
certain users may prefer to continue using narrow pointers. IMO wide
TOAST pointers should be a table option. Whether the default for new
tables should be wide or narrow pointers is debatable.

In another discussion [1] we seem to agree that we also want to have
an ability to include a 32-bit dictionary_id to the TOAST pointers and
perhaps support more compression methods (ZSTD to name one). Besides
that it would be nice to have an ability to extend TOAST pointers in
the future without breaking the existing pointers. One possible
solution would be to add a varint feature bitmask to every pointer. So
we could add flags like TOAST_IS_WIDE, TOAST_HAS_DICTIONARY,
TOAST_UNKNOWN_FEATURE_FROM_2077, etc indefinitely.

I suggest we address all the current and future needs once and
completely refactor TOAST pointers rather than solving one problem at
a time. I believe this will be more beneficial for the community in
the long term.

Thoughts?

[1]: https://commitfest.postgresql.org/43/3626/

--
Best regards,
Aleksander Alekseev