Обсуждение: BUG #16497: old and new pg_controldata WAL segment sizes are invalid or do not match

Поиск
Список
Период
Сортировка

BUG #16497: old and new pg_controldata WAL segment sizes are invalid or do not match

От
PG Bug reporting form
Дата:
The following bug has been logged on the website:

Bug reference:      16497
Logged by:          Ram Pratatp maurya
Email address:      ram.maurya@lavainternational.in
PostgreSQL version: 11.0
Operating system:   RHEL 6
Description:

Hello team,

I am facing error during upgrade from postgresql -11 to 12.
can you please suggest what is solution.
.................................................................................................................
"bash-4.1$ /usr/pgsql-12/bin/pg_upgrade --old-datadir
/var/lib/pgsql/11/data/ --new-datadir /var/lib/pgsql/12/data/ --old-bindir
/usr/pgsql-11/bin/ --new-bindir /usr/pgsql-12/bin/ --check
Performing Consistency Checks
-----------------------------
Checking cluster versions                                   ok

old and new pg_controldata WAL segment sizes are invalid or do not match
Failure, exiting
"
regards,
ram ptatap


Re: BUG #16497: old and new pg_controldata WAL segment sizes areinvalid or do not match

От
Michael Paquier
Дата:
On Wed, Jun 17, 2020 at 09:00:23PM +0000, PG Bug reporting form wrote:
> I am facing error during upgrade from postgresql -11 to 12.
> can you please suggest what is solution.

As the error message is telling, you need to make the WAL segment
sizes of the old and new clusters match.  This can be defined at
server initialization time using initdb with a default of 16MB, so
that's not a bug from us, but something you need to take care of
within your upgrade process.
--
Michael

Вложения

RE: BUG #16497: old and new pg_controldata WAL segment sizes areinvalid or do not match

От
Ram Pratap Maurya
Дата:
Dear Michael,

Currently I hav installed postgres12 version , need to uninstall and reinstall again.
Or any command that we change current installation wal segment size.

Below location we have installed 12 version.

/var/lib/pgsql/12/data


Regards,
Ram Pratap.

-----Original Message-----
From: Michael Paquier [mailto:michael@paquier.xyz]
Sent: 18 June 2020 13:35
To: Ram Pratap Maurya; pgsql-bugs@lists.postgresql.org
Subject: Re: BUG #16497: old and new pg_controldata WAL segment sizes are invalid or do not match

On Wed, Jun 17, 2020 at 09:00:23PM +0000, PG Bug reporting form wrote:
> I am facing error during upgrade from postgresql -11 to 12.
> can you please suggest what is solution.

As the error message is telling, you need to make the WAL segment sizes of the old and new clusters match.  This can be
definedat server initialization time using initdb with a default of 16MB, so that's not a bug from us, but something
youneed to take care of within your upgrade process. 
--
Michael



Re: BUG #16497: old and new pg_controldata WAL segment sizes areinvalid or do not match

От
Jeff Janes
Дата:
On Thu, Jun 18, 2020 at 4:05 AM Michael Paquier <michael@paquier.xyz> wrote:
On Wed, Jun 17, 2020 at 09:00:23PM +0000, PG Bug reporting form wrote:
> I am facing error during upgrade from postgresql -11 to 12.
> can you please suggest what is solution.

As the error message is telling, you need to make the WAL segment
sizes of the old and new clusters match.  This can be defined at
server initialization time using initdb with a default of 16MB, so
that's not a bug from us, but something you need to take care of
within your upgrade process.

Since wal-segsize is changeable with pg_resetwal since v11, and pg_upgrade is already calling pg_resetwal, shouldn't pg_upgrade ideally just deal with this situation automatically by allowing the upgrade to also change this value, rather than forcing the user to make them match manually?
 
Cheers,

Jeff

Re: BUG #16497: old and new pg_controldata WAL segment sizes areinvalid or do not match

От
Bruce Momjian
Дата:
On Thu, Jun 18, 2020 at 10:18:21AM -0400, Jeff Janes wrote:
> On Thu, Jun 18, 2020 at 4:05 AM Michael Paquier <michael@paquier.xyz> wrote:
> 
>     On Wed, Jun 17, 2020 at 09:00:23PM +0000, PG Bug reporting form wrote:
>     > I am facing error during upgrade from postgresql -11 to 12.
>     > can you please suggest what is solution.
> 
>     As the error message is telling, you need to make the WAL segment
>     sizes of the old and new clusters match.  This can be defined at
>     server initialization time using initdb with a default of 16MB, so
>     that's not a bug from us, but something you need to take care of
>     within your upgrade process.
> 
> 
> Since wal-segsize is changeable with pg_resetwal since v11, and pg_upgrade is
> already calling pg_resetwal, shouldn't pg_upgrade ideally just deal with this
> situation automatically by allowing the upgrade to also change this value,
> rather than forcing the user to make them match manually?

Uh, well, pg_upgrade is very conservative and fails if anything looks
odd, like different WAL sizes.  Since pg_upgrade doesn't even copy the
WAL from the old server, I am not sure why we require the WAL segment
size to match, but it might be so we can run pg_upgrade on replicas.

The bottom line is that pg_upgrade is complex enough, and having it try
to do more is not a wise approach.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EnterpriseDB                             https://enterprisedb.com

  The usefulness of a cup is in its emptiness, Bruce Lee




Re: BUG #16497: old and new pg_controldata WAL segment sizes are invalid or do not match

От
Tom Lane
Дата:
Jeff Janes <jeff.janes@gmail.com> writes:
> Since wal-segsize is changeable with pg_resetwal since v11, and pg_upgrade
> is already calling pg_resetwal, shouldn't pg_upgrade ideally just deal with
> this situation automatically by allowing the upgrade to also change this
> value, rather than forcing the user to make them match manually?

The issue is that this is an initdb parameter, and pg_upgrade expects you
to have already initdb'd the destination cluster.  We could redefine that,
perhaps, but it'd be a large change in how one uses pg_upgrade and would
certainly break a lot of scripts.

I'm aware that we could use pg_resetwal to deal with this one specific
initdb parameter, but I see no point in hacking around the problem for
just one parameter.  The general principle remains that you need to
initdb the target with the same settings you used for the source.

            regards, tom lane



Re: BUG #16497: old and new pg_controldata WAL segment sizes areinvalid or do not match

От
Bruce Momjian
Дата:
On Thu, Jun 18, 2020 at 11:16:50AM -0400, Tom Lane wrote:
> Jeff Janes <jeff.janes@gmail.com> writes:
> > Since wal-segsize is changeable with pg_resetwal since v11, and pg_upgrade
> > is already calling pg_resetwal, shouldn't pg_upgrade ideally just deal with
> > this situation automatically by allowing the upgrade to also change this
> > value, rather than forcing the user to make them match manually?
> 
> The issue is that this is an initdb parameter, and pg_upgrade expects you
> to have already initdb'd the destination cluster.  We could redefine that,
> perhaps, but it'd be a large change in how one uses pg_upgrade and would
> certainly break a lot of scripts.
> 
> I'm aware that we could use pg_resetwal to deal with this one specific
> initdb parameter, but I see no point in hacking around the problem for
> just one parameter.  The general principle remains that you need to
> initdb the target with the same settings you used for the source.

Yep, that's pretty much the issue.  :-)

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EnterpriseDB                             https://enterprisedb.com

  The usefulness of a cup is in its emptiness, Bruce Lee




Re: BUG #16497: old and new pg_controldata WAL segment sizes areinvalid or do not match

От
Stephen Frost
Дата:
Greetings,

* Bruce Momjian (bruce@momjian.us) wrote:
> On Thu, Jun 18, 2020 at 11:16:50AM -0400, Tom Lane wrote:
> > Jeff Janes <jeff.janes@gmail.com> writes:
> > > Since wal-segsize is changeable with pg_resetwal since v11, and pg_upgrade
> > > is already calling pg_resetwal, shouldn't pg_upgrade ideally just deal with
> > > this situation automatically by allowing the upgrade to also change this
> > > value, rather than forcing the user to make them match manually?
> >
> > The issue is that this is an initdb parameter, and pg_upgrade expects you
> > to have already initdb'd the destination cluster.  We could redefine that,
> > perhaps, but it'd be a large change in how one uses pg_upgrade and would
> > certainly break a lot of scripts.

I don't think we really need to redefine that, what it seems we'd need
to change is to not have pg_upgrade complain about things being
different between the two which don't actually matter to pg_upgrade.

> > I'm aware that we could use pg_resetwal to deal with this one specific
> > initdb parameter, but I see no point in hacking around the problem for
> > just one parameter.  The general principle remains that you need to
> > initdb the target with the same settings you used for the source.
>
> Yep, that's pretty much the issue.  :-)

Well, except that for pg_upgrade to do its job correctly, the two
clusters *don't* actually need be initdb'd exactly the same way, do they?

Sure, most options to initdb need to be the same between the old cluster
and the new cluster, but this specific option doesn't have to be, since
we require that it's a cleanly shut down cluster, so why are we
complaining about it if it's different..?

Thanks,

Stephen

Вложения

Re: BUG #16497: old and new pg_controldata WAL segment sizes areinvalid or do not match

От
Bruce Momjian
Дата:
On Thu, Jun 18, 2020 at 11:41:53AM -0400, Stephen Frost wrote:
> Sure, most options to initdb need to be the same between the old cluster
> and the new cluster, but this specific option doesn't have to be, since
> we require that it's a cleanly shut down cluster, so why are we
> complaining about it if it's different..?

Did you not read my previous email that we might have added this so we
can upgrade replicas?  I am sure I can dig out the commit that added
this and find the original cause, but running pg_upgrade on replicas is
enough of a reason to me.  Yeah, we could add a flag to pg_upgrade to
allow this if you are not upgrading replicas, but why bother?  It might
even work if you create the new replicas with the same WAL segment size,
but why add complexity for pg_upgrade, which is already complex enough.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EnterpriseDB                             https://enterprisedb.com

  The usefulness of a cup is in its emptiness, Bruce Lee




Re: BUG #16497: old and new pg_controldata WAL segment sizes areinvalid or do not match

От
Stephen Frost
Дата:
Greetings,

* Bruce Momjian (bruce@momjian.us) wrote:
> On Thu, Jun 18, 2020 at 11:41:53AM -0400, Stephen Frost wrote:
> > Sure, most options to initdb need to be the same between the old cluster
> > and the new cluster, but this specific option doesn't have to be, since
> > we require that it's a cleanly shut down cluster, so why are we
> > complaining about it if it's different..?
>
> Did you not read my previous email that we might have added this so we
> can upgrade replicas?

I don't see how this option is related to dealing with replicas..?

> I am sure I can dig out the commit that added
> this and find the original cause, but running pg_upgrade on replicas is
> enough of a reason to me.

pg_upgrade can't be run on replicas, so I don't understand what you're
referring to here..  The 'upgrading replicas' process that involves
rsync also requires everything to have been shut down cleanly.

> Yeah, we could add a flag to pg_upgrade to
> allow this if you are not upgrading replicas, but why bother?  It might
> even work if you create the new replicas with the same WAL segment size,
> but why add complexity for pg_upgrade, which is already complex enough.

Users already have to deal with various options that need to be
configured to match up between the primary and replicas, so this really
seems like it's entirely independent of pg_upgrade and isn't something
pg_upgrade needs to be worrying about..

Thanks,

Stephen

Вложения

Re: BUG #16497: old and new pg_controldata WAL segment sizes areinvalid or do not match

От
Jeff Janes
Дата:
On Thu, Jun 18, 2020 at 11:16 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
Jeff Janes <jeff.janes@gmail.com> writes:
> Since wal-segsize is changeable with pg_resetwal since v11, and pg_upgrade
> is already calling pg_resetwal, shouldn't pg_upgrade ideally just deal with
> this situation automatically by allowing the upgrade to also change this
> value, rather than forcing the user to make them match manually?

The issue is that this is an initdb parameter, and pg_upgrade expects you
to have already initdb'd the destination cluster.  We could redefine that,
perhaps, but it'd be a large change in how one uses pg_upgrade and would
certainly break a lot of scripts.

I'm aware that we could use pg_resetwal to deal with this one specific
initdb parameter, but I see no point in hacking around the problem for
just one parameter.  The general principle remains that you need to
initdb the target with the same settings you used for the source.

Since you mention it, now that -B is not necessary (inferred from where pg_upgrade itself is found), I have certainly thought it would also be nice if -D could point to a non-existent or empty directory, and have it do the initdb for you.  It would be nice to have it figure out the correct settings of -E, -U, --lc-collate, --lc-ctype and whatever else is needed to make --check pass, rather than doing it all manually (and one at a time, since it stops at the first error).  It doesn't seem like this, or the previously described change, would break any scripts which currently work.  It might cause some currently broken ones to work in ways that are unexpected, though.
 
Cheers,

Jeff

Re: BUG #16497: old and new pg_controldata WAL segment sizes areinvalid or do not match

От
Bruce Momjian
Дата:
On Thu, Jun 18, 2020 at 11:55:37AM -0400, Stephen Frost wrote:
> Greetings,
> 
> * Bruce Momjian (bruce@momjian.us) wrote:
> > On Thu, Jun 18, 2020 at 11:41:53AM -0400, Stephen Frost wrote:
> > > Sure, most options to initdb need to be the same between the old cluster
> > > and the new cluster, but this specific option doesn't have to be, since
> > > we require that it's a cleanly shut down cluster, so why are we
> > > complaining about it if it's different..?
> > 
> > Did you not read my previous email that we might have added this so we
> > can upgrade replicas?
> 
> I don't see how this option is related to dealing with replicas..?
> 
> > I am sure I can dig out the commit that added
> > this and find the original cause, but running pg_upgrade on replicas is
> > enough of a reason to me.
> 
> pg_upgrade can't be run on replicas, so I don't understand what you're
> referring to here..  The 'upgrading replicas' process that involves
> rsync also requires everything to have been shut down cleanly.

Oh, I forgot we copy the WAL from the primary and don't run initdb on
the standbys, so it might work, but looking at the pg_upgrade code, I
see:

    /* now reset the wal archives in the new cluster */
    prep_status("Resetting WAL archives");
    exec_prog(UTILITY_LOG_FILE, NULL, true, true,
    /* use timeline 1 to match controldata and no WAL history file */
              "\"%s/pg_resetwal\" -l 00000001%s \"%s\"", new_cluster.bindir,
              old_cluster.controldata.nextxlogfile + 8,
              new_cluster.pgdata);

So, while we don't copy over the old WAL, we do assume the WAL will be
at the same segment from the old cluster.  I think this is because of
the LSN stored on user data pages.  pg_resetwal doesn't seem to care
about that, so maybe pg_upgrade doesn't need to either.  I don't know.

> > Yeah, we could add a flag to pg_upgrade to
> > allow this if you are not upgrading replicas, but why bother?  It might
> > even work if you create the new replicas with the same WAL segment size,
> > but why add complexity for pg_upgrade, which is already complex enough.
> 
> Users already have to deal with various options that need to be
> configured to match up between the primary and replicas, so this really
> seems like it's entirely independent of pg_upgrade and isn't something
> pg_upgrade needs to be worrying about..

Do you know why we require this step?

    https://www.postgresql.org/docs/12/pgupgrade.html
    
    Also, change wal_level to replica in the postgresql.conf file on
    the new primary cluster.

The other modes don't work?  I see this C comment:

     * We unconditionally start/stop the new server because pg_resetwal -o set
     * wal_level to 'minimum'.  If the user is upgrading standby servers using
     * the rsync instructions, they will need pg_upgrade to write its final
     * WAL record showing wal_level as 'replica'.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EnterpriseDB                             https://enterprisedb.com

  The usefulness of a cup is in its emptiness, Bruce Lee




Re: BUG #16497: old and new pg_controldata WAL segment sizes areinvalid or do not match

От
Bruce Momjian
Дата:
On Thu, Jun 18, 2020 at 12:14:25PM -0400, Jeff Janes wrote:
> On Thu, Jun 18, 2020 at 11:16 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
> 
>     Jeff Janes <jeff.janes@gmail.com> writes:
>     > Since wal-segsize is changeable with pg_resetwal since v11, and
>     pg_upgrade
>     > is already calling pg_resetwal, shouldn't pg_upgrade ideally just deal
>     with
>     > this situation automatically by allowing the upgrade to also change this
>     > value, rather than forcing the user to make them match manually?
> 
>     The issue is that this is an initdb parameter, and pg_upgrade expects you
>     to have already initdb'd the destination cluster.  We could redefine that,
>     perhaps, but it'd be a large change in how one uses pg_upgrade and would
>     certainly break a lot of scripts.
> 
>     I'm aware that we could use pg_resetwal to deal with this one specific
>     initdb parameter, but I see no point in hacking around the problem for
>     just one parameter.  The general principle remains that you need to
>     initdb the target with the same settings you used for the source.
> 
> 
> Since you mention it, now that -B is not necessary (inferred from where
> pg_upgrade itself is found), I have certainly thought it would also be nice if
> -D could point to a non-existent or empty directory, and have it do the initdb
> for you.  It would be nice to have it figure out the correct settings of -E,
> -U, --lc-collate, --lc-ctype and whatever else is needed to make --check pass,
> rather than doing it all manually (and one at a time, since it stops at the
> first error).  It doesn't seem like this, or the previously described change,
> would break any scripts which currently work.  It might cause some currently
> broken ones to work in ways that are unexpected, though.

Yes, it would be nice, but there is also the requirement of adjusting
postgresql.conf to match the old cluster.  I am not sure we can get away
without that step, but looking at how people automate pg_upgrade would
help determine that.

Basically, if pg_upgrade ran initdb, we would need to automate steps 5-8
here:

    https://www.postgresql.org/docs/12/pgupgrade.html

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EnterpriseDB                             https://enterprisedb.com

  The usefulness of a cup is in its emptiness, Bruce Lee




Re: BUG #16497: old and new pg_controldata WAL segment sizes areinvalid or do not match

От
Stephen Frost
Дата:
Greetings,

* Bruce Momjian (bruce@momjian.us) wrote:
> On Thu, Jun 18, 2020 at 11:55:37AM -0400, Stephen Frost wrote:
> > * Bruce Momjian (bruce@momjian.us) wrote:
> > > On Thu, Jun 18, 2020 at 11:41:53AM -0400, Stephen Frost wrote:
> > > > Sure, most options to initdb need to be the same between the old cluster
> > > > and the new cluster, but this specific option doesn't have to be, since
> > > > we require that it's a cleanly shut down cluster, so why are we
> > > > complaining about it if it's different..?
> > >
> > > Did you not read my previous email that we might have added this so we
> > > can upgrade replicas?
> >
> > I don't see how this option is related to dealing with replicas..?
> >
> > > I am sure I can dig out the commit that added
> > > this and find the original cause, but running pg_upgrade on replicas is
> > > enough of a reason to me.
> >
> > pg_upgrade can't be run on replicas, so I don't understand what you're
> > referring to here..  The 'upgrading replicas' process that involves
> > rsync also requires everything to have been shut down cleanly.
>
> Oh, I forgot we copy the WAL from the primary and don't run initdb on
> the standbys, so it might work, but looking at the pg_upgrade code, I
> see:
>
>     /* now reset the wal archives in the new cluster */
>     prep_status("Resetting WAL archives");
>     exec_prog(UTILITY_LOG_FILE, NULL, true, true,
>     /* use timeline 1 to match controldata and no WAL history file */
>               "\"%s/pg_resetwal\" -l 00000001%s \"%s\"", new_cluster.bindir,
>               old_cluster.controldata.nextxlogfile + 8,
>               new_cluster.pgdata);
>
> So, while we don't copy over the old WAL, we do assume the WAL will be
> at the same segment from the old cluster.  I think this is because of
> the LSN stored on user data pages.  pg_resetwal doesn't seem to care
> about that, so maybe pg_upgrade doesn't need to either.  I don't know.

Yeah, this would need to be adjusted, since we're passing in an actual WAL
filename there rather than an LSN.

> > > Yeah, we could add a flag to pg_upgrade to
> > > allow this if you are not upgrading replicas, but why bother?  It might
> > > even work if you create the new replicas with the same WAL segment size,
> > > but why add complexity for pg_upgrade, which is already complex enough.
> >
> > Users already have to deal with various options that need to be
> > configured to match up between the primary and replicas, so this really
> > seems like it's entirely independent of pg_upgrade and isn't something
> > pg_upgrade needs to be worrying about..
>
> Do you know why we require this step?
>
>     https://www.postgresql.org/docs/12/pgupgrade.html
>
>     Also, change wal_level to replica in the postgresql.conf file on
>     the new primary cluster.

Well, we'll need wal_level to be at least replica if we're going to have
replicas streaming from the primary..

> The other modes don't work?  I see this C comment:
>
>      * We unconditionally start/stop the new server because pg_resetwal -o set
>      * wal_level to 'minimum'.  If the user is upgrading standby servers using
>      * the rsync instructions, they will need pg_upgrade to write its final
>      * WAL record showing wal_level as 'replica'.

Yes, the new system needs to have replica (or higher) as the wal_level,
but that doesn't depend on the WAL segment size.

Thanks,

Stephen

Вложения

Re: BUG #16497: old and new pg_controldata WAL segment sizes areinvalid or do not match

От
Bruce Momjian
Дата:
On Thu, Jun 18, 2020 at 01:42:41PM -0400, Stephen Frost wrote:
> > > > Yeah, we could add a flag to pg_upgrade to
> > > > allow this if you are not upgrading replicas, but why bother?  It might
> > > > even work if you create the new replicas with the same WAL segment size,
> > > > but why add complexity for pg_upgrade, which is already complex enough.
> > > 
> > > Users already have to deal with various options that need to be
> > > configured to match up between the primary and replicas, so this really
> > > seems like it's entirely independent of pg_upgrade and isn't something
> > > pg_upgrade needs to be worrying about..
> > 
> > Do you know why we require this step?
> > 
> >     https://www.postgresql.org/docs/12/pgupgrade.html
> >     
> >     Also, change wal_level to replica in the postgresql.conf file on
> >     the new primary cluster.
> 
> Well, we'll need wal_level to be at least replica if we're going to have
> replicas streaming from the primary..

But how do they have replicas if wal_level = minimum?  Also, why not
higher replication levels?  Should we adjust that doc text?

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EnterpriseDB                             https://enterprisedb.com

  The usefulness of a cup is in its emptiness, Bruce Lee




Re: BUG #16497: old and new pg_controldata WAL segment sizes areinvalid or do not match

От
Stephen Frost
Дата:
Greetings,

* Bruce Momjian (bruce@momjian.us) wrote:
> On Thu, Jun 18, 2020 at 01:42:41PM -0400, Stephen Frost wrote:
> > > > > Yeah, we could add a flag to pg_upgrade to
> > > > > allow this if you are not upgrading replicas, but why bother?  It might
> > > > > even work if you create the new replicas with the same WAL segment size,
> > > > > but why add complexity for pg_upgrade, which is already complex enough.
> > > >
> > > > Users already have to deal with various options that need to be
> > > > configured to match up between the primary and replicas, so this really
> > > > seems like it's entirely independent of pg_upgrade and isn't something
> > > > pg_upgrade needs to be worrying about..
> > >
> > > Do you know why we require this step?
> > >
> > >     https://www.postgresql.org/docs/12/pgupgrade.html
> > >
> > >     Also, change wal_level to replica in the postgresql.conf file on
> > >     the new primary cluster.
> >
> > Well, we'll need wal_level to be at least replica if we're going to have
> > replicas streaming from the primary..
>
> But how do they have replicas if wal_level = minimum?  Also, why not
> higher replication levels?  Should we adjust that doc text?

I think the comment is saying that pg_resetwal will rewrite the
pg_control with a WAL level of minimal and that's the issue and why
the server needs to be brought up with a higher WAL level temporarily,
so that pg_control gets updated, for the new cluster.

Of course, before pg_upgrade is run, the old cluster would need to be up
and running with a wal_level higher than minimal in order to have
replicas in the first place, but what we're really talking about here is
the new, upgraded, cluster.

I do think the doc could probably say replica 'or higher'.

Thanks,

Stephen

Вложения

Re: BUG #16497: old and new pg_controldata WAL segment sizes areinvalid or do not match

От
Bruce Momjian
Дата:
On Thu, Jun 18, 2020 at 02:11:14PM -0400, Stephen Frost wrote:
> Greetings,
> 
> * Bruce Momjian (bruce@momjian.us) wrote:
> > On Thu, Jun 18, 2020 at 01:42:41PM -0400, Stephen Frost wrote:
> > > > > > Yeah, we could add a flag to pg_upgrade to
> > > > > > allow this if you are not upgrading replicas, but why bother?  It might
> > > > > > even work if you create the new replicas with the same WAL segment size,
> > > > > > but why add complexity for pg_upgrade, which is already complex enough.
> > > > > 
> > > > > Users already have to deal with various options that need to be
> > > > > configured to match up between the primary and replicas, so this really
> > > > > seems like it's entirely independent of pg_upgrade and isn't something
> > > > > pg_upgrade needs to be worrying about..
> > > > 
> > > > Do you know why we require this step?
> > > > 
> > > >     https://www.postgresql.org/docs/12/pgupgrade.html
> > > >     
> > > >     Also, change wal_level to replica in the postgresql.conf file on
> > > >     the new primary cluster.
> > > 
> > > Well, we'll need wal_level to be at least replica if we're going to have
> > > replicas streaming from the primary..
> > 
> > But how do they have replicas if wal_level = minimum?  Also, why not
> > higher replication levels?  Should we adjust that doc text?
> 
> I think the comment is saying that pg_resetwal will rewrite the
> pg_control with a WAL level of minimal and that's the issue and why
> the server needs to be brought up with a higher WAL level temporarily,
> so that pg_control gets updated, for the new cluster.
> 
> Of course, before pg_upgrade is run, the old cluster would need to be up
> and running with a wal_level higher than minimal in order to have
> replicas in the first place, but what we're really talking about here is
> the new, upgraded, cluster.
> 
> I do think the doc could probably say replica 'or higher'.

OK, I will work on that, thanks.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EnterpriseDB                             https://enterprisedb.com

  The usefulness of a cup is in its emptiness, Bruce Lee




Re: BUG #16497: old and new pg_controldata WAL segment sizes areinvalid or do not match

От
Michael Paquier
Дата:
On Thu, Jun 18, 2020 at 02:29:43PM -0400, Bruce Momjian wrote:
> On Thu, Jun 18, 2020 at 02:11:14PM -0400, Stephen Frost wrote:
>> I do think the doc could probably say replica 'or higher'.

+1.  Sounds good to me to just use "or higher" here.  The docs insist
on the concept of hierarchy for values of wal_level.

> OK, I will work on that, thanks.

Thanks.
--
Michael

Вложения

RE: BUG #16497: old and new pg_controldata WAL segment sizes areinvalid or do not match

От
Ram Pratap Maurya
Дата:
Dear All,

any impact of server. If we Can reset manually postgres old version (11) WAL segment size .


Regards,
Ram Pratap.
Lava International Limited.
Tel+  91-120-4637148



-----Original Message-----
From: Bruce Momjian [mailto:bruce@momjian.us]
Sent: 19 June 2020 00:00
To: Stephen Frost
Cc: Tom Lane; Jeff Janes; Michael Paquier; Ram Pratap Maurya; pgsql-bugs
Subject: Re: BUG #16497: old and new pg_controldata WAL segment sizes are invalid or do not match

On Thu, Jun 18, 2020 at 02:11:14PM -0400, Stephen Frost wrote:
> Greetings,
>
> * Bruce Momjian (bruce@momjian.us) wrote:
> > On Thu, Jun 18, 2020 at 01:42:41PM -0400, Stephen Frost wrote:
> > > > > > Yeah, we could add a flag to pg_upgrade to allow this if you
> > > > > > are not upgrading replicas, but why bother?  It might even
> > > > > > work if you create the new replicas with the same WAL
> > > > > > segment size, but why add complexity for pg_upgrade, which is already complex enough.
> > > > >
> > > > > Users already have to deal with various options that need to
> > > > > be configured to match up between the primary and replicas, so
> > > > > this really seems like it's entirely independent of pg_upgrade
> > > > > and isn't something pg_upgrade needs to be worrying about..
> > > >
> > > > Do you know why we require this step?
> > > >
> > > >     https://www.postgresql.org/docs/12/pgupgrade.html
> > > >
> > > >     Also, change wal_level to replica in the postgresql.conf file on
> > > >     the new primary cluster.
> > >
> > > Well, we'll need wal_level to be at least replica if we're going
> > > to have replicas streaming from the primary..
> >
> > But how do they have replicas if wal_level = minimum?  Also, why not
> > higher replication levels?  Should we adjust that doc text?
>
> I think the comment is saying that pg_resetwal will rewrite the
> pg_control with a WAL level of minimal and that's the issue and why
> the server needs to be brought up with a higher WAL level temporarily,
> so that pg_control gets updated, for the new cluster.
>
> Of course, before pg_upgrade is run, the old cluster would need to be
> up and running with a wal_level higher than minimal in order to have
> replicas in the first place, but what we're really talking about here
> is the new, upgraded, cluster.
>
> I do think the doc could probably say replica 'or higher'.

OK, I will work on that, thanks.

--
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EnterpriseDB                             https://enterprisedb.com

  The usefulness of a cup is in its emptiness, Bruce Lee




RE: BUG #16497: old and new pg_controldata WAL segment sizes are invalid or do not match

От
Ram Pratap Maurya
Дата:

Dear All,

 

We have reinstall postgres-12 version but when I run below command  its showing error again.

Please suggest what I do to resolve this issue.

 

Command:

/usr/pgsql-12/bin/pg_upgrade --old-datadir /var/lib/pgsql/11/data/ --new-datadir /var/lib/pgsql/12/data/ --old-bindir /usr/pgsql-11/bin/ --new-bindir /usr/pgsql-12/bin/ --check

 

 

Regards,

Ram Pratap.

Lava International Limited.

Tel+  91-120-4637148

 

 

 

-----Original Message-----
From: Bruce Momjian [mailto:bruce@momjian.us]
Sent: 19 June 2020 00:00
To: Stephen Frost
Cc: Tom Lane; Jeff Janes; Michael Paquier; Ram Pratap Maurya; pgsql-bugs
Subject: Re: BUG #16497: old and new pg_controldata WAL segment sizes are invalid or do not match

 

On Thu, Jun 18, 2020 at 02:11:14PM -0400, Stephen Frost wrote:

> Greetings,

>

> * Bruce Momjian (bruce@momjian.us) wrote:

> > On Thu, Jun 18, 2020 at 01:42:41PM -0400, Stephen Frost wrote:

> > > > > > Yeah, we could add a flag to pg_upgrade to allow this if you

> > > > > > are not upgrading replicas, but why bother?  It might even

> > > > > > work if you create the new replicas with the same WAL

> > > > > > segment size, but why add complexity for pg_upgrade, which is already complex enough.

> > > > >

> > > > > Users already have to deal with various options that need to

> > > > > be configured to match up between the primary and replicas, so

> > > > > this really seems like it's entirely independent of pg_upgrade

> > > > > and isn't something pg_upgrade needs to be worrying about..

> > > >

> > > > Do you know why we require this step?

> > > >

> > > > https://www.postgresql.org/docs/12/pgupgrade.html

> > > >

> > > > Also, change wal_level to replica in the postgresql.conf file on

> > > > the new primary cluster.

> > >

> > > Well, we'll need wal_level to be at least replica if we're going

> > > to have replicas streaming from the primary..

> >

> > But how do they have replicas if wal_level = minimum?  Also, why not

> > higher replication levels?  Should we adjust that doc text?

>

> I think the comment is saying that pg_resetwal will rewrite the

> pg_control with a WAL level of minimal and that's the issue and why

> the server needs to be brought up with a higher WAL level temporarily,

> so that pg_control gets updated, for the new cluster.

>

> Of course, before pg_upgrade is run, the old cluster would need to be

> up and running with a wal_level higher than minimal in order to have

> replicas in the first place, but what we're really talking about here

> is the new, upgraded, cluster.

>

> I do think the doc could probably say replica 'or higher'.

 

OK, I will work on that, thanks.

 

--

  Bruce Momjian  <bruce@momjian.us>        https://momjian.us

  EnterpriseDB                             https://enterprisedb.com

 

  The usefulness of a cup is in its emptiness, Bruce Lee

 

Вложения

Re: BUG #16497: old and new pg_controldata WAL segment sizes areinvalid or do not match

От
Bruce Momjian
Дата:
On Fri, Jun 19, 2020 at 05:20:21AM +0000, Ram Pratap Maurya wrote:
> Dear All,
> 
> any impact of server. If we Can reset manually postgres old version (11) WAL segment size . 

You can change the wal segment size _after_ running pg_upgrade, but for
the upgrade, they have to match.  Use an initdb flag so they do.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EnterpriseDB                             https://enterprisedb.com

  The usefulness of a cup is in its emptiness, Bruce Lee




Re: BUG #16497: old and new pg_controldata WAL segment sizes areinvalid or do not match

От
Bruce Momjian
Дата:
On Fri, Jun 19, 2020 at 09:48:55AM +0900, Michael Paquier wrote:
> On Thu, Jun 18, 2020 at 02:29:43PM -0400, Bruce Momjian wrote:
> > On Thu, Jun 18, 2020 at 02:11:14PM -0400, Stephen Frost wrote:
> >> I do think the doc could probably say replica 'or higher'.
> 
> +1.  Sounds good to me to just use "or higher" here.  The docs insist
> on the concept of hierarchy for values of wal_level.

Instead of mentioning a specific wal_level "or higher", I decided to
just mention the WAL level _not_ to use;  patch attached.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EnterpriseDB                             https://enterprisedb.com

  The usefulness of a cup is in its emptiness, Bruce Lee


Вложения

Re: BUG #16497: old and new pg_controldata WAL segment sizes areinvalid or do not match

От
Michael Paquier
Дата:
On Sat, Jun 27, 2020 at 05:26:10PM -0400, Bruce Momjian wrote:
> Instead of mentioning a specific wal_level "or higher", I decided to
> just mention the WAL level _not_ to use;  patch attached.

Sounds fine to me, thanks!
--
Michael

Вложения

RE: BUG #16497: old and new pg_controldata WAL segment sizes are invalid or do not match

От
Ram Pratap Maurya
Дата:

Thanks to all for you we have solved “old and new pg_controldata WAL segment sizes are invalid or do not match ” issue during upgrade.

 

 

Below command details

 

PWD   : /usr/pgsql-12/bin

1.            Stop the server 12

2.            Remove all data from location /var/lib/pgsql/12/data

3.            Go to /usr/pgsql-12/bin

4.            Run below command

./initdb -D /var/lib/pgsql/12/data -U postgres --wal-segsize=64

 

5. run Upgrade command :

 

/usr/pgsql-12/bin/pg_upgrade --old-datadir /var/lib/pgsql/11/data/ --new-datadir /var/lib/pgsql/12/data/ --old-bindir /usr/pgsql-11/bin/ --new-bindir /usr/pgsql-12/bin/

 

 

Regards,

Ram Pratap.

Lava International Limited.

Tel+  91-120-4637148

 

 

 

-----Original Message-----
From: Michael Paquier [mailto:michael@paquier.xyz]
Sent: 29 June 2020 13:05
To: Bruce Momjian
Cc: Stephen Frost; Tom Lane; Jeff Janes; Ram Pratap Maurya; pgsql-bugs
Subject: Re: BUG #16497: old and new pg_controldata WAL segment sizes are invalid or do not match

 

On Sat, Jun 27, 2020 at 05:26:10PM -0400, Bruce Momjian wrote:

> Instead of mentioning a specific wal_level "or higher", I decided to

> just mention the WAL level _not_ to use;  patch attached.

 

Sounds fine to me, thanks!

--

Michael

Вложения

Re: BUG #16497: old and new pg_controldata WAL segment sizes are invalid or do not match

От
Bruce Momjian
Дата:
On Sat, Jun 27, 2020 at 05:26:10PM -0400, Bruce Momjian wrote:
> On Fri, Jun 19, 2020 at 09:48:55AM +0900, Michael Paquier wrote:
> > On Thu, Jun 18, 2020 at 02:29:43PM -0400, Bruce Momjian wrote:
> > > On Thu, Jun 18, 2020 at 02:11:14PM -0400, Stephen Frost wrote:
> > >> I do think the doc could probably say replica 'or higher'.
> > 
> > +1.  Sounds good to me to just use "or higher" here.  The docs insist
> > on the concept of hierarchy for values of wal_level.
> 
> Instead of mentioning a specific wal_level "or higher", I decided to
> just mention the WAL level _not_ to use;  patch attached.

Patch applied through 9.5.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EnterpriseDB                             https://enterprisedb.com

  The usefulness of a cup is in its emptiness, Bruce Lee