Обсуждение: 2PC support for pglogical
Hi. Here is proof-of-concept version of two phase commit support for logical replication. There are some changes in core postgres,pglogical_output and pglogical extensions. I’ve used version from 2nd Quadrant repo, pglogical branch and rebasedit to current head. Some notes about this patch: * LWLockAssign() was deleted, so I changed that to new locks tranche api. * Seems that only reliable way to get GID during replay of commit/rollback prepared is to force postgres to write GID incorresponding records, otherwise we can lose correspondence between xid and gid if we are replaying data from wal senderwhile transaction was commited some time ago. So i’ve changed postgres to write gid’s not only on prepare, but alsoon commit/rollback prepared. That should be done only in logical level, but now I just want to here some other opinionson that. * Abort prepared xlog record also lack database information. Normally logical decoding just cleans reorder buffer when facingabort, but in case of 2PC we should send it to callbacks anyway. So I’ve added that info to abort records. * Prepare emits xlog record with TwoPhaseFileHeader in it and that structure is the same as xl_xact_parsed_commit, but withsome fields renamed. Probably that is just due to historical reasons. It is possible to change PREPARE to write ordinarycommit records with some flag and then use the same infrastructure to parse it. So DecodePrepare/DecodeCommit canbe done with the same function. --- Stas Kelvich Postgres Professional: http://www.postgrespro.com Russian Postgres Company
Вложения
On 10 March 2016 at 22:50, Stas Kelvich <s.kelvich@postgrespro.ru> wrote:
Hi.
Here is proof-of-concept version of two phase commit support for logical replication.
I missed this when you posted it, so sorry for the late response.
I've read through this but not tested it yet. I really appreciate you doing it, it's been on my wishlist/backburner for ages.
On reading through the patch I noticed that there doesn't seem to be any consideration of locking. The prepared xact can still hold strong locks on catalogs. How's that handled? I think Robert's group locking stuff is what we'll want here - for a prepared xact we can join the prepared xact as a group lock member so we inherit its locks. Right now if you try DDL in a prepared xact I suspect it'll break.
On a more minor note, I think the pglogical_output and pglogical changes should be a separate patch to the patch to core PostgreSQL. Keep them separate. "git format-patch" on a tree is good for this.
A few misc comments on my first read-through:
Shouldn't PGLOGICAL_COMMIT etc be an enum { } ?
+/*
+ * ParsePrepareRecord
+ */
isn't a super-helpful comment.
Not sure what this means:
+ /* Ignoring abortrels? */
+ bufptr += MAXALIGN(hdr->nabortrels * sizeof(RelFileNode));
typo:
+ ... shortcut for coomiting bare xact ...
DecodePrepare seems to be mostly a copy of DecodeCommit. Can that be done with less duplication?
Not sure I understand the rationale of "bare xact" as terminology/naming, e.g. ReorderBufferCommitBareXact . I guess you're getting at the fact that it's just a commit or rollback, not data. Phase2 ? 2PC?
* Seems that only reliable way to get GID during replay of commit/rollback prepared is to force postgres to write GID in corresponding records, otherwise we can lose correspondence between xid and gid if we are replaying data from wal sender while transaction was commited some time ago. So i’ve changed postgres to write gid’s not only on prepare, but also on commit/rollback prepared. That should be done only in logical level, but now I just want to here some other opinions on that.
That sounds sensible.
* Abort prepared xlog record also lack database information. Normally logical decoding just cleans reorder buffer when facing abort, but in case of 2PC we should send it to callbacks anyway. So I’ve added that info to abort records.
I was wondering what that was. Again, seems sensible, though I'm not totally convinced of the naming.
On Wed, Mar 23, 2016 at 1:44 AM, Craig Ringer <craig@2ndquadrant.com> wrote: > On 10 March 2016 at 22:50, Stas Kelvich <s.kelvich@postgrespro.ru> wrote: >> Hi. >> >> Here is proof-of-concept version of two phase commit support for logical >> replication. > > I missed this when you posted it, so sorry for the late response. > > I've read through this but not tested it yet. I really appreciate you doing > it, it's been on my wishlist/backburner for ages. > > On reading through the patch I noticed that there doesn't seem to be any > consideration of locking. The prepared xact can still hold strong locks on > catalogs. How's that handled? I think Robert's group locking stuff is what > we'll want here - for a prepared xact we can join the prepared xact as a > group lock member so we inherit its locks. Right now if you try DDL in a > prepared xact I suspect it'll break. I have grave doubts about that approach. It's not impossible it could be right, but I wouldn't bet the farm on it. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company
> On 24 Mar 2016, at 17:03, Robert Haas <robertmhaas@gmail.com> wrote: > > On Wed, Mar 23, 2016 at 1:44 AM, Craig Ringer <craig@2ndquadrant.com> wrote: >> On 10 March 2016 at 22:50, Stas Kelvich <s.kelvich@postgrespro.ru> wrote: >>> Hi. >>> >>> Here is proof-of-concept version of two phase commit support for logical >>> replication. >> >> I missed this when you posted it, so sorry for the late response. >> >> I've read through this but not tested it yet. I really appreciate you doing >> it, it's been on my wishlist/backburner for ages. >> >> On reading through the patch I noticed that there doesn't seem to be any >> consideration of locking. The prepared xact can still hold strong locks on >> catalogs. How's that handled? I think Robert's group locking stuff is what >> we'll want here - for a prepared xact we can join the prepared xact as a >> group lock member so we inherit its locks. Right now if you try DDL in a >> prepared xact I suspect it'll break. > > I have grave doubts about that approach. It's not impossible it could > be right, but I wouldn't bet the farm on it. > I think all the locking already handled properly by creating dummy backend in PGPROC, as it done in usual postgres 2pc implementation. Just checked DDL with following scenario: * prepare tx that inserts a row in table on master * execute DROP TABLE on pglogical slave * commit prepared on master Now it behaves as expected — slave blocks DROP TABLE until commit prepared on master. --- Stas Kelvich Postgres Professional: http://www.postgrespro.com Russian Postgres Company
On 24 March 2016 at 22:20, Stas Kelvich <s.kelvich@postgrespro.ru> wrote:
I think all the locking already handled properly by creating dummy backend in PGPROC, as it done in usual postgres 2pc implementation.
On the downstream, yes. But what about the decoding, reorder buffer and output plugin? They access the relcache etc. If you changed the structure of a table in a prepared xact you have to make sure you get that right.
Just checked DDL with following scenario:
* prepare tx that inserts a row in table on master
* execute DROP TABLE on pglogical slave
* commit prepared on master
Now it behaves as expected — slave blocks DROP TABLE until commit prepared on master.
Try doing DDL in the prepared xact on the upstream side, especially things that make structural changes to tables. Test both things that do full-table rewrites (alter table ... add column ... default not null) and things that don't (alter table ... drop column ; alter table ... add column ...; without not null or default).