Обсуждение: register/unregister standby Re: Synchronous replication
On Tue, Aug 10, 2010 at 5:58 AM, Heikki Linnakangas <heikki.linnakangas@enterprisedb.com> wrote: > On 05/08/10 17:14, Fujii Masao wrote: >> >> I'm thinking to make users register and unregister each standbys via SQL >> functions like register_standby() and unregister_standby(): > > The register/unregister facility should be accessible from the streaming > replication connection, so that you don't need to connect to any particular > database in addition to the streaming connection. Probably I've not understood your point correctly yet. I think that the advantage of registering standbys is that we can specify which WAL files the master has to keep for the upcoming standby. IMO, it's usually called together with pg_start_backup as follows: SELECT register_standby('foo', pg_start_backup()) This requests the master keep to all the WAL files following the backup starting location which pg_start_backup returns. Now we can do that by using wal_keep_segments, but it's not easy to set because it's difficult to predict how many WAL files the standby will require. So I've thought that the register/unregister facility should be used from the normal client connection. Why do you think it should be accessible from the SR connection? Regards, -- Fujii Masao NIPPON TELEGRAPH AND TELEPHONE CORPORATION NTT Open Source Software Center
On 30/08/10 15:14, Fujii Masao wrote: > I think that the advantage of registering standbys is that we can > specify which WAL files the master has to keep for the upcoming > standby. IMO, it's usually called together with pg_start_backup > as follows: > > SELECT register_standby('foo', pg_start_backup()) > > This requests the master keep to all the WAL files following the > backup starting location which pg_start_backup returns. Hmm, that's clever. I was thinking that you'd initialize the standby from an existing backup, and in that context the standby would not need to connect to the master except via the replication connection. To take a base backup, you'll need not only that but also access to the filesystem in the master, ie. shell access. There's been some talk of being able to stream a base backup over the replication connection too, which would be extremely handy. And that makes my point even stronger that registering a standby should be possible via the replication connection. Of course, you could well expose the functionality as both a built-in function and a command in replication mode, so this detail isn't really that important right now. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com
Heikki Linnakangas <heikki.linnakangas@enterprisedb.com> writes: > Hmm, that's clever. I was thinking that you'd initialize the standby from an > existing backup, and in that context the standby would not need to connect > to the master except via the replication connection. To take a base backup, > you'll need not only that but also access to the filesystem in the master, > ie. shell access. In fact you don't need shell access here, it's rather easy to stream the base backup from the libpq connection, as implemented here : http://github.com/dimitri/pg_basebackup > There's been some talk of being able to stream a base backup over the > replication connection too, which would be extremely handy. Yes please ! :) -- dim
On Thu, Sep 2, 2010 at 6:41 PM, Dimitri Fontaine <dfontaine@hi-media.com> wrote: > In fact you don't need shell access here, it's rather easy to stream the > base backup from the libpq connection, as implemented here : > > http://github.com/dimitri/pg_basebackup > >> There's been some talk of being able to stream a base backup over the >> replication connection too, which would be extremely handy. > > Yes please ! :) One issue of the base backup function is that the operation will be a long transaction. So, non-transactional special commands, as like as VACUUM, would be better in terms of performance. For example, CREATE or ALTER REPLICATION. Of course, function-based approach is more flexible and less invasive to the SQL parser. There are trade-offs. -- Itagaki Takahiro
On 30 August 2010 13:14, Fujii Masao <masao.fujii@gmail.com> wrote: > I think that the advantage of registering standbys is that we can > specify which WAL files the master has to keep for the upcoming > standby. IMO, it's usually called together with pg_start_backup > as follows: > > SELECT register_standby('foo', pg_start_backup()) > > This requests the master keep to all the WAL files following the > backup starting location which pg_start_backup returns. Now we > can do that by using wal_keep_segments, but it's not easy to set > because it's difficult to predict how many WAL files the standby > will require. +1 I don't like the idea of having to guess how many WAL files you think you'll need to keep around. And if these standby instances have to register, could there be a view to list subscriber information? -- Thom Brown Twitter: @darkixion IRC (freenode): dark_ixion Registered Linux user: #516935
Itagaki Takahiro <itagaki.takahiro@gmail.com> writes: >> http://github.com/dimitri/pg_basebackup >> >>> There's been some talk of being able to stream a base backup over the >>> replication connection too, which would be extremely handy. >> >> Yes please ! :) > > One issue of the base backup function is that the operation will > be a long transaction. So, non-transactional special commands, > as like as VACUUM, would be better in terms of performance. > For example, CREATE or ALTER REPLICATION. Well, you still need to stream the data to the client in a format it will understand. Would that be the plan of your command proposal? > Of course, function-based approach is more flexible and > less invasive to the SQL parser. There are trade-offs. Well that was easier for a proof-of-concept at least. -- Dimitri Fontaine PostgreSQL DBA, Architecte
On Thu, Sep 2, 2010 at 7:54 PM, Dimitri Fontaine <dfontaine@hi-media.com> wrote: >> One issue of the base backup function is that the operation will >> be a long transaction. So, non-transactional special commands, >> as like as VACUUM, would be better in terms of performance. >> For example, CREATE or ALTER REPLICATION. > > Well, you still need to stream the data to the client in a format it > will understand. True, but using libpq connection might be not the most important thing. The most simplest proof-of-concept might be system("rsync") in the function ;-) > Would that be the plan of your command proposal? What I meant was function-based maintenance does not work well in some cases. I heard before pg_start_backup( no-fast-checkpoint ) caused table bloating problem because it was a long transaction for 20+ minutes. The backup function would have the similar issue. -- Itagaki Takahiro