Hi,
I am trying to use logical replication mechanism in implementation of PostgreSQL multimaster and faced with one
conceptualproblem.
Originally logical replication was intended to support asynchronous replication. In this case applying changes by
singleprocess should not be a bottleneck.
But if we are using distributed transaction manager to provide global consistency, then applying transaction by one
worker leads to very bad performance and what is worser: cause unintended serialization of transactions, which is not
takenin account by
distributed deadlock detection algorithm and so can cause
undetected deadlocks.
So I have implemented pool of background workers which can apply transactions concurrently.
It works and shows acceptable performance. But now I am thinking about HA and tracking origin LSNs which are needed to
correctlyspecify slot position in case of recovery. And there is a problem: as far as I understand to correctly record
originLSN in WAL
and advance slot position it is necessary to setup session
using replorigin_session_setup. It is not so convenient in case of using pool of background workers, because we have to
setupsession for each commit.
But the main problem is that for each slot session can be associated only with one process:
else if (curstate->acquired_by != 0) { ereport(ERROR,
(errcode(ERRCODE_OBJECT_IN_USE), errmsg("replication identifier %d is already active for PID %d",
curstate->roident, curstate->acquired_by))); }
Which once again means that there can be only one process applying changes.
To provide correct state of replication node it is necessary to enforce that each logical replication record is
replayedexactly once: we should not loose some change or try to apply it twice. So operation of recording original LSN
positionin WAL and
adjusting slot should be atomic. And during recovery we should restore slot current position based on the origin values
extracted from WAL. I wonder if it can be done using current logical replication mechanism when changes of each slot
areapplied by
more than one process? Or the only alternative is to write/read origin LSNs in WAL myself, for example using custom WAL
records?
Thanks in advance!
--
Konstantin Knizhnik
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company