Hi hackers!
We need community feedback on previously discussed topic [1].
There are some long-live issues in Postgres related to the TOAST mechanics, like [2].
Some time ago we already proposed a set of patches with an API allowing to plug in
different TOAST implementations into a live database. The patch set introduced a lot
of code and was quite crude in some places, so after several implementations we decided
to try to implement it in the production environment for further check-up.
The main idea behind pluggable TOAST is make it possible to easily plug in and use different
implementations of large values storage, preserving existing mechanics to keep backward
compatibilitну provide easy Postgres-way give users alternative mechanics for storing large
column values in a more effective way - we already have custom and very effective (up to tens
and even hundreds of times faster) TOAST implementations for bytea and JSONb data types.
As we see it - Pluggable TOAST proposes
1) changes in TOAST pointer itself, extending it to store custom data - current limitations
of TOAST pointer were discussed in [1] and [4];
2) API which allows calls of custom TOAST implementations for certain table columns and
(a topic for discussion) certain datatypes.
Custom TOAST could be also used in a not so trivial way - for example, limited columnar storage could be easily implemented and plugged in without heavy core modifications
of implementation of Pluggable Storage (Table Access Methods), preserving existing data
and database structure, be upgraded, replicated and so on.
Any thoughts and proposals are welcome.
-- Regards,
Nikita Malakhov
Postgres Professional
The Russian Postgres Company