Обсуждение: pg_upgrade support is broken?
Hi While reading the setup script, I noticed that to initdb the new cluster for pg_upgrade runs the following: # Perform initdb on the new server $PGENGINE/postgresql92-setup initdb Also, PREVPGENGINE isn't referenced; instead the pg_upgrade code uses PGPREVENGINE . This suggests that pg_upgrade support via the initscript has been broken throughout 9.3 and is still broken in 9.4, even for direct upgrades from the immediately previous major. It also uses 'su' instead of 'runuser', so I think it'd have issues on SELinux. It doesn't look like upgrades from other than the immediately previous major could ever have worked: - Support for specifying a non-default version to upgrade seems broken. Lots of the code continues to use $PREVMAJORVERSION despite it not being set if the user passes a $OLD_SERVICE_NAME argument. - it assumes that the previous binaries are in /usr/pgsql-$PREVMAJORVERSION/bin but they aren't if the user passed a $OLD_SERVICE_NAME other than that of $PREVMAJORVERSION - The issue with PREVPGENGINE vs PGPREVENGINE above Also, $OLD_SERVICE_NAME must be the basename of the initscript/service control file, e.g. postgresql-9.3 , not just "9.3". This isn't super obvious from the help. Support for upgrading from the immediately prev version looks to be fixable by replacing: # Perform initdb on the new server $PGENGINE/postgresql92-setup initdb with # Perform initdb on the new server $PGENGINE/postgresql${PREVMAJORVERSION/.}-setup initdb + using "$SU" instead of "su" Fixing upgrading from earlier versions also needs PREVMAJORVERSION to be overridden if the user specifies a control file explicitly and a bunch of other fixes. I attach a majorly reworked script for your review and comment. I haven't done much testing yet. It requires the rpm builds script to sub in some new variables: PREVMAJORVERSION=%{prevmajorversion} PGUNITNAME=%{unitname} PGPACKAGENAME=%{name} so a wrapper like: substitute_vars() { # Insert RPM variables into shell scripts sed -e 's|^PGVERSION=.*$|PGVERSION=%{version}|' \ -e 's|^PGENGINE=.*$|PGENGINE=%{pgbaseinstdir}/bin|' \ -e 's|^PGUNITNAME=.*$|PGUNITNAME=%{unitname}|' \ -e 's|^PREVMAJORVERSION=.*$|PREVMAJORVERSION=%{prevmajorversion}|' \ -e 's|^PGPACKAGENAME=.*$|PGPACKAGENAME=%{name}|' \ -e 's|^PGDOCDIR=.*$|PGDOCDIR=%{_pkgdocdir}|' \ < "$1" > "$2" } # prep the setup script, including insertion of some values it needs substitute_vars "%{SOURCE17}" "%{name}-setup" might be useful. BTW, the README.rpm-dist looks like it's a victim of some copy-and-paste too; see: ``` In 9.1+, the RPMs also support in-place upgrade from th immediately previous major release. Currently, you can upgrade in-place from 9.2.x to 9.4.x. Just run: $ /usr/pgsql-9.4/bin/postgresql94-setup upgrade Please note that 9.2 and 9.4 contrib RPMs needs to be installed for this feature to work. ``` That should be "9.3" not "9.2" and there's a typo in "th[e]". Anyway. Thoughts on the updated scripts? I've combined the sysv and systemd scripts to reduce duplication. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services
Вложения
Hi Craig, On Fri, 2014-08-01 at 22:15 +0800, Craig Ringer wrote: > Anyway. Thoughts on the updated scripts? I've combined the sysv and > systemd scripts to reduce duplication. What is "__RPM_PGENGINE_ " and other similar stuff in both of these patches? I am ok to apply these, once I have an understading how they are expanded into real paths or so. Cheers, -- Devrim GÜNDÜZ Principal Systems Engineer @ EnterpriseDB: http://www.enterprisedb.com PostgreSQL Danışmanı/Consultant, Red Hat Certified Engineer Twitter: @DevrimGunduz , @DevrimGunduzTR
Вложения
On 09/11/2014 05:09 AM, Devrim Gündüz wrote: > > Hi Craig, > > On Fri, 2014-08-01 at 22:15 +0800, Craig Ringer wrote: > >> Anyway. Thoughts on the updated scripts? I've combined the sysv and >> systemd scripts to reduce duplication. > > What is "__RPM_PGENGINE_ " and other similar stuff in both of these > patches? I am ok to apply these, once I have an understading how they > are expanded into real paths or so. Best hold off a little, I have a considerably cleaned up and updated version nearly ready, based off the BDR release packages with the BDR stuff stripped out. I'll move all the substitutions into var declarations at the top of each script for clarity. Should've done it earlier. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services
Hi, On Thu, 2014-09-11 at 12:11 +0800, Craig Ringer wrote: > Best hold off a little, I have a considerably cleaned up and updated > version nearly ready, based off the BDR release packages with the BDR > stuff stripped out. > > I'll move all the substitutions into var declarations at the top of > each script for clarity. Should've done it earlier. Thanks. Do you think you can complete this in next few weeks? I don't think 9.4.0 is too far away. Regards, -- Devrim GÜNDÜZ Principal Systems Engineer @ EnterpriseDB: http://www.enterprisedb.com PostgreSQL Danışmanı/Consultant, Red Hat Certified Engineer Twitter: @DevrimGunduz , @DevrimGunduzTR
Вложения
On 09/23/2014 06:42 PM, Devrim Gündüz wrote: > > Hi, > > On Thu, 2014-09-11 at 12:11 +0800, Craig Ringer wrote: >> Best hold off a little, I have a considerably cleaned up and >> updated version nearly ready, based off the BDR release packages >> with the BDR stuff stripped out. >> >> I'll move all the substitutions into var declarations at the top >> of each script for clarity. Should've done it earlier. > > Thanks. Do you think you can complete this in next few weeks? I > don't think 9.4.0 is too far away. Yes. It's been on my "tomorrow" list for days now, but urgent things keep popping up. I'll make an hour or two for it tomorrow one way or the other. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services
On Fri, 2014-08-01 at 22:15 +0800, Craig Ringer wrote: Hi, you may be interested about Pavel's recent work on postgresql-setup [1] that we are using in Fedora and planing to use in RHEL. There is a lot of work to support migration scenarios from different versions. [1] https://praiskup.fedorapeople.org/proj/postgresql-setup/ > Hi > > While reading the setup script, I noticed that to initdb the new > cluster > for pg_upgrade runs the following: > > # Perform initdb on the new server > $PGENGINE/postgresql92-setup initdb > > > Also, PREVPGENGINE isn't referenced; instead the pg_upgrade code uses > PGPREVENGINE . > > This suggests that pg_upgrade support via the initscript has been > broken > throughout 9.3 and is still broken in 9.4, even for direct upgrades > from > the immediately previous major. > > It also uses 'su' instead of 'runuser', so I think it'd have issues > on > SELinux. > > > > It doesn't look like upgrades from other than the immediately > previous > major could ever have worked: > > - Support for specifying a non-default version to upgrade seems > broken. > Lots of the code continues to use $PREVMAJORVERSION despite it not > being > set if the user passes a $OLD_SERVICE_NAME argument. > > - it assumes that the previous binaries are in > /usr/pgsql-$PREVMAJORVERSION/bin but they aren't if the user passed a > $OLD_SERVICE_NAME other than that of $PREVMAJORVERSION > > - The issue with PREVPGENGINE vs PGPREVENGINE above > > > > > Also, $OLD_SERVICE_NAME must be the basename of the > initscript/service > control file, e.g. postgresql-9.3 , not just "9.3". This isn't super > obvious from the help. > > > Support for upgrading from the immediately prev version looks to be > fixable by replacing: > > > # Perform initdb on the new server > $PGENGINE/postgresql92-setup initdb > > with > > # Perform initdb on the new server > $PGENGINE/postgresql${PREVMAJORVERSION/.}-setup initdb > > + using "$SU" instead of "su" > > > Fixing upgrading from earlier versions also needs PREVMAJORVERSION to > be > overridden if the user specifies a control file explicitly and a > bunch > of other fixes. > > I attach a majorly reworked script for your review and comment. I > haven't done much testing yet. > > It requires the rpm builds script to sub in some new variables: > > PREVMAJORVERSION=%{prevmajorversion} > PGUNITNAME=%{unitname} > PGPACKAGENAME=%{name} > > > so a wrapper like: > > substitute_vars() { > # Insert RPM variables into shell scripts > sed -e 's|^PGVERSION=.*$|PGVERSION=%{version}|' \ > -e 's|^PGENGINE=.*$|PGENGINE=%{pgbaseinstdir}/bin|' \ > -e 's|^PGUNITNAME=.*$|PGUNITNAME=%{unitname}|' \ > -e > 's|^PREVMAJORVERSION=.*$|PREVMAJORVERSION=%{prevmajorversion}|' \ > -e 's|^PGPACKAGENAME=.*$|PGPACKAGENAME=%{name}|' \ > -e 's|^PGDOCDIR=.*$|PGDOCDIR=%{_pkgdocdir}|' \ > < "$1" > "$2" > } > > # prep the setup script, including insertion of some values it needs > substitute_vars "%{SOURCE17}" "%{name}-setup" > > > > might be useful. > > > > > > > BTW, the README.rpm-dist looks like it's a victim of some copy-and > -paste > too; see: > > > ``` > In 9.1+, the RPMs also support in-place upgrade from th immediately > previous major release. Currently, you can upgrade in-place > from 9.2.x to 9.4.x. Just run: > $ /usr/pgsql-9.4/bin/postgresql94-setup upgrade > > Please note that 9.2 and 9.4 contrib RPMs needs to be installed for > this > feature to work. > ``` > > That should be "9.3" not "9.2" and there's a typo in "th[e]". > > > > Anyway. Thoughts on the updated scripts? I've combined the sysv and > systemd scripts to reduce duplication. >
On Wed, 2015-05-13 at 08:52 +0200, Jozef Mlich wrote: Sorry, I didn't noticed, that I am replying on old message .. > On Fri, 2014-08-01 at 22:15 +0800, Craig Ringer wrote: > > > Hi, > > you may be interested about Pavel's recent work on postgresql-setup > [1] > that we are using in Fedora and planing to use in RHEL. > > There is a lot of work to support migration scenarios from different > versions. > > [1] https://praiskup.fedorapeople.org/proj/postgresql-setup/ > > > > Hi > > > > While reading the setup script, I noticed that to initdb the new > > cluster > > for pg_upgrade runs the following: > > > > # Perform initdb on the new server > > $PGENGINE/postgresql92-setup initdb > > > > > > Also, PREVPGENGINE isn't referenced; instead the pg_upgrade code > > uses > > PGPREVENGINE . > > > > This suggests that pg_upgrade support via the initscript has been > > broken > > throughout 9.3 and is still broken in 9.4, even for direct upgrades > > > > from > > the immediately previous major. > > > > It also uses 'su' instead of 'runuser', so I think it'd have issues > > > > on > > SELinux. > > > > > > > > It doesn't look like upgrades from other than the immediately > > previous > > major could ever have worked: > > > > - Support for specifying a non-default version to upgrade seems > > broken. > > Lots of the code continues to use $PREVMAJORVERSION despite it not > > being > > set if the user passes a $OLD_SERVICE_NAME argument. > > > > - it assumes that the previous binaries are in > > /usr/pgsql-$PREVMAJORVERSION/bin but they aren't if the user passed > > a > > $OLD_SERVICE_NAME other than that of $PREVMAJORVERSION > > > > - The issue with PREVPGENGINE vs PGPREVENGINE above > > > > > > > > > > Also, $OLD_SERVICE_NAME must be the basename of the > > initscript/service > > control file, e.g. postgresql-9.3 , not just "9.3". This isn't > > super > > obvious from the help. > > > > > > Support for upgrading from the immediately prev version looks to be > > fixable by replacing: > > > > > > # Perform initdb on the new server > > $PGENGINE/postgresql92-setup initdb > > > > with > > > > # Perform initdb on the new server > > $PGENGINE/postgresql${PREVMAJORVERSION/.}-setup initdb > > > > + using "$SU" instead of "su" > > > > > > Fixing upgrading from earlier versions also needs PREVMAJORVERSION > > to > > be > > overridden if the user specifies a control file explicitly and a > > bunch > > of other fixes. > > > > I attach a majorly reworked script for your review and comment. I > > haven't done much testing yet. > > > > It requires the rpm builds script to sub in some new variables: > > > > PREVMAJORVERSION=%{prevmajorversion} > > PGUNITNAME=%{unitname} > > PGPACKAGENAME=%{name} > > > > > > so a wrapper like: > > > > substitute_vars() { > > # Insert RPM variables into shell scripts > > sed -e 's|^PGVERSION=.*$|PGVERSION=%{version}|' \ > > -e 's|^PGENGINE=.*$|PGENGINE=%{pgbaseinstdir}/bin|' \ > > -e 's|^PGUNITNAME=.*$|PGUNITNAME=%{unitname}|' \ > > -e > > 's|^PREVMAJORVERSION=.*$|PREVMAJORVERSION=%{prevmajorversion}|' \ > > -e 's|^PGPACKAGENAME=.*$|PGPACKAGENAME=%{name}|' \ > > -e 's|^PGDOCDIR=.*$|PGDOCDIR=%{_pkgdocdir}|' \ > > < "$1" > "$2" > > } > > > > # prep the setup script, including insertion of some values it > > needs > > substitute_vars "%{SOURCE17}" "%{name}-setup" > > > > > > > > might be useful. > > > > > > > > > > > > > > BTW, the README.rpm-dist looks like it's a victim of some copy-and > > -paste > > too; see: > > > > > > ``` > > In 9.1+, the RPMs also support in-place upgrade from th immediately > > previous major release. Currently, you can upgrade in-place > > from 9.2.x to 9.4.x. Just run: > > $ /usr/pgsql-9.4/bin/postgresql94-setup upgrade > > > > Please note that 9.2 and 9.4 contrib RPMs needs to be installed for > > > > this > > feature to work. > > ``` > > > > That should be "9.3" not "9.2" and there's a typo in "th[e]". > > > > > > > > Anyway. Thoughts on the updated scripts? I've combined the sysv and > > systemd scripts to reduce duplication. > > > > >
Hi, On Wed, 2015-05-13 at 08:53 +0200, Jozef Mlich wrote: > Sorry, I didn't noticed, that I am replying on old message .. It is ok. Our upgrade script is very broken already. A new approach would be good. I'll check. Regards, -- Devrim GÜNDÜZ Principal Systems Engineer @ EnterpriseDB: http://www.enterprisedb.com PostgreSQL Danışmanı/Consultant, Red Hat Certified Engineer Twitter: @DevrimGunduz , @DevrimGunduzTR