Обсуждение: pg_upgrade: "pg_ctl failed to start the new server"
I'm trying to upgrade Postgres 9.0 to 9.1 with pg_upgrade. Both versions are installed from the PGDG Yum repo: -bash-4.1$ /usr/pgsql-9.0/bin/postgres -V postgres (PostgreSQL) 9.0.8 -bash-4.1$ /usr/pgsql-9.1/bin/postgres -V postgres (PostgreSQL) 9.1.4 I can successfully start and connect to both 9.0 and 9.1. When I run pg_upgrade, it fails with the error "pg_ctl failed to start the new server," however it apparently does start the new server (output below). Any ideas? This is a freshly installed CentOS 6.2 machine with the 9.0 DB copied over from another machine via streaming replication (which I have since ended and the server is now standalone). Thanks. # su - postgres -bash-4.1$ ps ax | grep postg 4354 pts/0 S 0:00 su - postgres 4389 pts/0 S+ 0:00 grep postg -bash-4.1$ /usr/pgsql-9.1/bin/pg_upgrade -c -b /usr/pgsql-9.0/bin/ -B /usr/pgsql-9.1/bin/ -d /var/lib/pgsql/9.0/data/ -D /var/lib/pgsql/9.1/data/ Performing Consistency Checks ----------------------------- Checking current, bin, and data directories ok Checking cluster versions ok Checking database user is a superuser ok Checking for prepared transactions ok Checking for reg* system oid user data types ok Checking for contrib/isn with bigint-passing mismatch ok pg_ctl failed to start the new server Failure, exiting -bash-4.1$ ps ax | grep postg 4354 pts/0 S 0:00 su - postgres 4445 pts/0 S 0:00 /usr/pgsql-9.1/bin/postgres -D /var/lib/pgsql/9.1/data -p 5432 -b 4446 ? Ss 0:00 postgres: logger process 4448 ? Ss 0:00 postgres: writer process 4449 ? Ss 0:00 postgres: wal writer process 4450 ? Ss 0:00 postgres: stats collector process 4453 pts/0 S+ 0:00 grep postg
On Wed, Jun 13, 2012 at 11:19:41AM -0400, Evan D. Hoffman wrote: > I'm trying to upgrade Postgres 9.0 to 9.1 with pg_upgrade. Both > versions are installed from the PGDG Yum repo: > > -bash-4.1$ /usr/pgsql-9.0/bin/postgres -V > postgres (PostgreSQL) 9.0.8 > -bash-4.1$ /usr/pgsql-9.1/bin/postgres -V > postgres (PostgreSQL) 9.1.4 > > I can successfully start and connect to both 9.0 and 9.1. When I run > pg_upgrade, it fails with the error "pg_ctl failed to start the new > server," however it apparently does start the new server (output > below). Any ideas? This is a freshly installed CentOS 6.2 machine > with the 9.0 DB copied over from another machine via streaming > replication (which I have since ended and the server is now > standalone). Have you looked at the logs from pg_upgrade? Does "pg_ctl -w start" work? My guess is that the -w (wait) is failing for some reason. We did improve the -w handling in PG 9.1 so I didn't think there were any failure cases, but you might have found a new one. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. +
On Wed, Jun 13, 2012 at 10:41:37PM -0400, Evan D. Hoffman wrote: > Actually I found the solution right after I sent that email (of > course): > > https://wiki-bsse.ethz.ch/download/attachments/55283107/PostgreSQL_9_M > aintenance_Backup_and_Recovery_final.docx > > Has to do with the order in which shared libs are loaded when both 9.0 > and 9.1 are installed. Renaming the config file as described in that > doc resolved it. I didn't find out which file you renamed. Why would the server show as started while pg_ctl -w returned an error? I am guessing that you could not connect somehow. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. +
Woops. The file that had to be renamed was /etc/ld.so.conf.d/postgresql-9.0-libs.conf => postgresql-9.old-libs.conf . Prior to rename, ldconfig output was: # ldconfig -p | grep pq libpqwalreceiver.so (libc6) => /usr/pgsql-9.0/lib/libpqwalreceiver.so libpqwalreceiver.so (libc6) => /usr/pgsql-9.1/lib/libpqwalreceiver.so libpq.so.5 (libc6) => /usr/pgsql-9.0/lib/libpq.so.5 libpq.so.5 (libc6) => /usr/pgsql-9.1/lib/libpq.so.5 libpq.so (libc6) => /usr/pgsql-9.0/lib/libpq.so libpq.so (libc6) => /usr/pgsql-9.1/lib/libpq.so After the rename: # cd /etc/ld.so.conf.d # mv postgresql-9.0-libs.conf postgresql-9.old-libs.conf # ldconfig # ldconfig -p | grep pq libpqwalreceiver.so (libc6) => /usr/pgsql-9.1/lib/libpqwalreceiver.so libpqwalreceiver.so (libc6) => /usr/pgsql-9.0/lib/libpqwalreceiver.so libpq.so.5 (libc6) => /usr/pgsql-9.1/lib/libpq.so.5 libpq.so.5 (libc6) => /usr/pgsql-9.0/lib/libpq.so.5 libpq.so (libc6) => /usr/pgsql-9.1/lib/libpq.so libpq.so (libc6) => /usr/pgsql-9.0/lib/libpq.so So it appears the problem was inability to connect, although pg_upgrade reported that it couldn't start the server (I assume ability to connect is how it determines whether or not the server was started). On Wed, Jun 13, 2012 at 10:49 PM, Bruce Momjian <bruce@momjian.us> wrote: > On Wed, Jun 13, 2012 at 10:41:37PM -0400, Evan D. Hoffman wrote: >> Actually I found the solution right after I sent that email (of >> course): >> >> https://wiki-bsse.ethz.ch/download/attachments/55283107/PostgreSQL_9_M >> aintenance_Backup_and_Recovery_final.docx >> >> Has to do with the order in which shared libs are loaded when both 9.0 >> and 9.1 are installed. Renaming the config file as described in that >> doc resolved it. > > I didn't find out which file you renamed. Why would the server show as > started while pg_ctl -w returned an error? I am guessing that you could > not connect somehow. > > -- > Bruce Momjian <bruce@momjian.us> http://momjian.us > EnterpriseDB http://enterprisedb.com > > + It's impossible for everything to be true. +
Actually I found the solution right after I sent that email (of course): https://wiki-bsse.ethz.ch/download/attachments/55283107/PostgreSQL_9_Maintenance_Backup_and_Recovery_final.docx Has to do with the order in which shared libs are loaded when both 9.0 and 9.1 are installed. Renaming the config file asdescribed in that doc resolved it. On Jun 13, 2012, at 10:26 PM, Bruce Momjian <bruce@momjian.us> wrote: > On Wed, Jun 13, 2012 at 11:19:41AM -0400, Evan D. Hoffman wrote: >> I'm trying to upgrade Postgres 9.0 to 9.1 with pg_upgrade. Both >> versions are installed from the PGDG Yum repo: >> >> -bash-4.1$ /usr/pgsql-9.0/bin/postgres -V >> postgres (PostgreSQL) 9.0.8 >> -bash-4.1$ /usr/pgsql-9.1/bin/postgres -V >> postgres (PostgreSQL) 9.1.4 >> >> I can successfully start and connect to both 9.0 and 9.1. When I run >> pg_upgrade, it fails with the error "pg_ctl failed to start the new >> server," however it apparently does start the new server (output >> below). Any ideas? This is a freshly installed CentOS 6.2 machine >> with the 9.0 DB copied over from another machine via streaming >> replication (which I have since ended and the server is now >> standalone). > > Have you looked at the logs from pg_upgrade? Does "pg_ctl -w start" > work? My guess is that the -w (wait) is failing for some reason. We > did improve the -w handling in PG 9.1 so I didn't think there were any > failure cases, but you might have found a new one. > > -- > Bruce Momjian <bruce@momjian.us> http://momjian.us > EnterpriseDB http://enterprisedb.com > > + It's impossible for everything to be true. +
On Thu, Jun 14, 2012 at 08:03:49AM -0400, Evan D. Hoffman wrote: > So it appears the problem was inability to connect, although > pg_upgrade reported that it couldn't start the server (I assume > ability to connect is how it determines whether or not the server was > started). Ah, OK, so it was the connection that failed. I wasn't aware pg_ctl start could fail in that case (we added PGping), but obviously it can. The attached patch applied to PG 9.2 and current will give a more appropriate error message saying it might be a connection problem. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. +