Обсуждение: pgsql: Have pg_receivexlog always send an invalid log position in statu
Have pg_receivexlog always send an invalid log position in status messages This prevents pg_basebackup and pg_receivexlog from becoming a synchronous standby in case 'write' is used for synchronous_commit. Fujii Masao Branch ------ master Details ------- http://git.postgresql.org/pg/commitdiff/d7ea9193d18d29bea82b30604cdd087e90c5ee27 Modified Files -------------- src/bin/pg_basebackup/receivelog.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)
Re: pgsql: Have pg_receivexlog always send an invalid log position in statu
От
Heikki Linnakangas
Дата:
On 09.02.2012 15:14, Magnus Hagander wrote: > Have pg_receivexlog always send an invalid log position in status messages > > This prevents pg_basebackup and pg_receivexlog from becoming a synchronous > standby in case 'write' is used for synchronous_commit. It's not completely crazy to use pg_receivexlog as a synchronous standby. It provides the zero-loss property like a real standby does, ie. if the master dies after sending the WAL to pg_receivexlog, that transaction is safe in the archive. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com
On Thu, Feb 9, 2012 at 11:35 AM, Heikki Linnakangas <heikki.linnakangas@enterprisedb.com> wrote: > On 09.02.2012 15:14, Magnus Hagander wrote: >> >> Have pg_receivexlog always send an invalid log position in status messages >> >> This prevents pg_basebackup and pg_receivexlog from becoming a synchronous >> standby in case 'write' is used for synchronous_commit. > > > It's not completely crazy to use pg_receivexlog as a synchronous standby. It > provides the zero-loss property like a real standby does, ie. if the master > dies after sending the WAL to pg_receivexlog, that transaction is safe in > the archive. > what if we ignore the application_name "pg_receivexlog" for a sync rep and give pg_receivexlog an option to set another application_name? that will avoid the problem for normal cases and still accept it for a sync rep if that is what we really want -- Jaime Casanova www.2ndQuadrant.com Professional PostgreSQL: Soporte 24x7 y capacitación
Re: pgsql: Have pg_receivexlog always send an invalid log position in statu
От
Magnus Hagander
Дата:
On Thu, Feb 9, 2012 at 17:35, Heikki Linnakangas <heikki.linnakangas@enterprisedb.com> wrote: > On 09.02.2012 15:14, Magnus Hagander wrote: >> >> Have pg_receivexlog always send an invalid log position in status messages >> >> This prevents pg_basebackup and pg_receivexlog from becoming a synchronous >> standby in case 'write' is used for synchronous_commit. > > > It's not completely crazy to use pg_receivexlog as a synchronous standby. It > provides the zero-loss property like a real standby does, ie. if the master > dies after sending the WAL to pg_receivexlog, that transaction is safe in > the archive. Yes, but as I stated in the email in the thread that the patch was posted in, I think this should not be the default behaviour, but it should be available as a commandline option, or something along that line. -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/
On Fri, Feb 10, 2012 at 8:14 AM, Magnus Hagander <magnus@hagander.net> wrote: > On Thu, Feb 9, 2012 at 17:35, Heikki Linnakangas > <heikki.linnakangas@enterprisedb.com> wrote: >> On 09.02.2012 15:14, Magnus Hagander wrote: >>> >>> Have pg_receivexlog always send an invalid log position in status messages >>> >>> This prevents pg_basebackup and pg_receivexlog from becoming a synchronous >>> standby in case 'write' is used for synchronous_commit. >> >> >> It's not completely crazy to use pg_receivexlog as a synchronous standby. It >> provides the zero-loss property like a real standby does, ie. if the master >> dies after sending the WAL to pg_receivexlog, that transaction is safe in >> the archive. > > Yes, but as I stated in the email in the thread that the patch was > posted in, I think this should not be the default behaviour, but it > should be available as a commandline option, or something along that > line. Even if we make that the default behavior, pg_receivexlog doesn't work as a sync standby unless synchronous_standby_names is set to "pg_receivexlog" or "*". There is little risk that we make that the default, I think... No? Anyway, to consider pg_receivexlog as a sync standby, we need to change it so that its status report includes the valid write and flush positions, and so that it replies as soon as it writes or flushes the received WAL, like real sync standby does. Otherwise, the master has to wait for the status report interval (which is specified in -s or --statusint option of pg_receivexlog). The proposed change would increase the frequency for pg_receivexlog to send back the report very much. Which might be a problem. For people who want to avoid such frequent reports, we might need to introduce the option which specifies whether frequent report is allowed or not. Regards, -- Fujii Masao NIPPON TELEGRAPH AND TELEPHONE CORPORATION NTT Open Source Software Center
Re: pgsql: Have pg_receivexlog always send an invalid log position in statu
От
Magnus Hagander
Дата:
On Fri, Feb 10, 2012 at 04:34, Fujii Masao <masao.fujii@gmail.com> wrote: > On Fri, Feb 10, 2012 at 8:14 AM, Magnus Hagander <magnus@hagander.net> wrote: >> On Thu, Feb 9, 2012 at 17:35, Heikki Linnakangas >> <heikki.linnakangas@enterprisedb.com> wrote: >>> On 09.02.2012 15:14, Magnus Hagander wrote: >>>> >>>> Have pg_receivexlog always send an invalid log position in status messages >>>> >>>> This prevents pg_basebackup and pg_receivexlog from becoming a synchronous >>>> standby in case 'write' is used for synchronous_commit. >>> >>> >>> It's not completely crazy to use pg_receivexlog as a synchronous standby. It >>> provides the zero-loss property like a real standby does, ie. if the master >>> dies after sending the WAL to pg_receivexlog, that transaction is safe in >>> the archive. >> >> Yes, but as I stated in the email in the thread that the patch was >> posted in, I think this should not be the default behaviour, but it >> should be available as a commandline option, or something along that >> line. > > Even if we make that the default behavior, pg_receivexlog doesn't work as > a sync standby unless synchronous_standby_names is set to "pg_receivexlog" > or "*". There is little risk that we make that the default, I think... No? We discussed this at some previous time, and since it's fairly common for people to use "*" - in fact, I believe it's what most people do. Which would lead to unintended consequences. I guess we could document that very clearly in the docs of that parameter... > Anyway, to consider pg_receivexlog as a sync standby, we need to change it > so that its status report includes the valid write and flush > positions, and so that > it replies as soon as it writes or flushes the received WAL, like real > sync standby > does. Otherwise, the master has to wait for the status report interval (which is > specified in -s or --statusint option of pg_receivexlog). Yes, that's what I suggested be controled by a separate parameter. Having it sync standby and only send status reports every now and then seems like a really bad idea. > The proposed change would increase the frequency for pg_receivexlog to send > back the report very much. Which might be a problem. For people who want to > avoid such frequent reports, we might need to introduce the option > which specifies > whether frequent report is allowed or not. Exactly my point... -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/
On Wed, Feb 15, 2012 at 6:44 PM, Magnus Hagander <magnus@hagander.net> wrote: > On Fri, Feb 10, 2012 at 04:34, Fujii Masao <masao.fujii@gmail.com> wrote: >> On Fri, Feb 10, 2012 at 8:14 AM, Magnus Hagander <magnus@hagander.net> wrote: >>> On Thu, Feb 9, 2012 at 17:35, Heikki Linnakangas >>> <heikki.linnakangas@enterprisedb.com> wrote: >>>> On 09.02.2012 15:14, Magnus Hagander wrote: >>>>> >>>>> Have pg_receivexlog always send an invalid log position in status messages >>>>> >>>>> This prevents pg_basebackup and pg_receivexlog from becoming a synchronous >>>>> standby in case 'write' is used for synchronous_commit. >>>> >>>> >>>> It's not completely crazy to use pg_receivexlog as a synchronous standby. It >>>> provides the zero-loss property like a real standby does, ie. if the master >>>> dies after sending the WAL to pg_receivexlog, that transaction is safe in >>>> the archive. >>> >>> Yes, but as I stated in the email in the thread that the patch was >>> posted in, I think this should not be the default behaviour, but it >>> should be available as a commandline option, or something along that >>> line. >> >> Even if we make that the default behavior, pg_receivexlog doesn't work as >> a sync standby unless synchronous_standby_names is set to "pg_receivexlog" >> or "*". There is little risk that we make that the default, I think... No? > > We discussed this at some previous time, and since it's fairly common > for people to use "*" - in fact, I believe it's what most people do. > Which would lead to unintended consequences. I guess we could document > that very clearly in the docs of that parameter... > > >> Anyway, to consider pg_receivexlog as a sync standby, we need to change it >> so that its status report includes the valid write and flush >> positions, and so that >> it replies as soon as it writes or flushes the received WAL, like real >> sync standby >> does. Otherwise, the master has to wait for the status report interval (which is >> specified in -s or --statusint option of pg_receivexlog). > > Yes, that's what I suggested be controled by a separate parameter. > > Having it sync standby and only send status reports every now and then > seems like a really bad idea. > > >> The proposed change would increase the frequency for pg_receivexlog to send >> back the report very much. Which might be a problem. For people who want to >> avoid such frequent reports, we might need to introduce the option >> which specifies >> whether frequent report is allowed or not. > > Exactly my point... Okay, barring objection, I will make the patch. But before making the patch, could you commit the patch which fix the incorrect handling of the timeout in pg_receivexlog? Or I should merge them into one patch? http://archives.postgresql.org/pgsql-hackers/2012-02/msg00257.php Regards, -- Fujii Masao NIPPON TELEGRAPH AND TELEPHONE CORPORATION NTT Open Source Software Center
Re: pgsql: Have pg_receivexlog always send an invalid log position in statu
От
Magnus Hagander
Дата:
On Wed, Feb 15, 2012 at 18:28, Fujii Masao <masao.fujii@gmail.com> wrote: > On Wed, Feb 15, 2012 at 6:44 PM, Magnus Hagander <magnus@hagander.net> wrote: >> On Fri, Feb 10, 2012 at 04:34, Fujii Masao <masao.fujii@gmail.com> wrote: >>> On Fri, Feb 10, 2012 at 8:14 AM, Magnus Hagander <magnus@hagander.net> wrote: >>>> On Thu, Feb 9, 2012 at 17:35, Heikki Linnakangas >>>> <heikki.linnakangas@enterprisedb.com> wrote: >>>>> On 09.02.2012 15:14, Magnus Hagander wrote: >>>>>> >>>>>> Have pg_receivexlog always send an invalid log position in status messages >>>>>> >>>>>> This prevents pg_basebackup and pg_receivexlog from becoming a synchronous >>>>>> standby in case 'write' is used for synchronous_commit. >>>>> >>>>> >>>>> It's not completely crazy to use pg_receivexlog as a synchronous standby. It >>>>> provides the zero-loss property like a real standby does, ie. if the master >>>>> dies after sending the WAL to pg_receivexlog, that transaction is safe in >>>>> the archive. >>>> >>>> Yes, but as I stated in the email in the thread that the patch was >>>> posted in, I think this should not be the default behaviour, but it >>>> should be available as a commandline option, or something along that >>>> line. >>> >>> Even if we make that the default behavior, pg_receivexlog doesn't work as >>> a sync standby unless synchronous_standby_names is set to "pg_receivexlog" >>> or "*". There is little risk that we make that the default, I think... No? >> >> We discussed this at some previous time, and since it's fairly common >> for people to use "*" - in fact, I believe it's what most people do. >> Which would lead to unintended consequences. I guess we could document >> that very clearly in the docs of that parameter... >> >> >>> Anyway, to consider pg_receivexlog as a sync standby, we need to change it >>> so that its status report includes the valid write and flush >>> positions, and so that >>> it replies as soon as it writes or flushes the received WAL, like real >>> sync standby >>> does. Otherwise, the master has to wait for the status report interval (which is >>> specified in -s or --statusint option of pg_receivexlog). >> >> Yes, that's what I suggested be controled by a separate parameter. >> >> Having it sync standby and only send status reports every now and then >> seems like a really bad idea. >> >> >>> The proposed change would increase the frequency for pg_receivexlog to send >>> back the report very much. Which might be a problem. For people who want to >>> avoid such frequent reports, we might need to introduce the option >>> which specifies >>> whether frequent report is allowed or not. >> >> Exactly my point... > > Okay, barring objection, I will make the patch. > > But before making the patch, could you commit the patch which fix the incorrect > handling of the timeout in pg_receivexlog? Or I should merge them into > one patch? > http://archives.postgresql.org/pgsql-hackers/2012-02/msg00257.php (Sorry about the delays) That one was sitting in my queue of "waiting for feedback". Did you have a comment on the topic of backwards compatibility raised in that thread? (and do comment in that thread in that case) And I think we're better off keeping them as separate patches, if nothing else that makes the reviews shorter :-) -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/