Обсуждение: pg_ctl only allows 12 parameters?

Поиск
Список
Период
Сортировка

pg_ctl only allows 12 parameters?

От
Ron Johnson
Дата:
PG 14.10

pg_ctl failed when I added a thirteenth parameter (log_temp_files).  Is this documented, and I missed it?

(The purpose is to optimize for a 30-thread pg_restore of a multi-TB 9.6 database.  Once it's finished, I stop/start, to enable "production" values like fsync=on.)

pg_ctl restart -wt999 -mfast
          -o '-c fsync=off' \
          -o '-c full_page_writes=off' \
          -o '-c log_statement=none' \
          -o '-c log_temp_files=100kB' \
          -o '-c max_wal_size=36GB' \
          -o '-c wal_level=minimal' \
          -o '-c max_wal_senders=0' \
          -o '-c wal_buffers=64MB' \
          -o '-c log_checkpoints=off' \
          -o '-c autovacuum=off'
pg_ctl: too many command-line arguments (first is "restart")

Re: pg_ctl only allows 12 parameters?

От
"David G. Johnston"
Дата:
On Monday, December 18, 2023, Ron Johnson <ronljohnsonjr@gmail.com> wrote:
PG 14.10

pg_ctl failed when I added a thirteenth parameter (log_temp_files).  Is this documented, and I missed it?

(The purpose is to optimize for a 30-thread pg_restore of a multi-TB 9.6 database.  Once it's finished, I stop/start, to enable "production" values like fsync=on.)

pg_ctl restart -wt999 -mfast
          -o '-c fsync=off' \
          -o '-c full_page_writes=off' \
          -o '-c log_statement=none' \
          -o '-c log_temp_files=100kB' \
          -o '-c max_wal_size=36GB' \
          -o '-c wal_level=minimal' \
          -o '-c max_wal_senders=0' \
          -o '-c wal_buffers=64MB' \
          -o '-c log_checkpoints=off' \
          -o '-c autovacuum=off'
pg_ctl: too many command-line arguments (first is "restart")

Arguments are not parameters.  Try putting the restart argument after all parameters.

Also, you are missing a line continuation backslash so you actually only supplied three arguments, where only 1 is accepted.

David J.
 

Re: pg_ctl only allows 12 parameters?

От
Ron Johnson
Дата:
On Mon, Dec 18, 2023 at 10:11 AM David G. Johnston <david.g.johnston@gmail.com> wrote:
On Monday, December 18, 2023, Ron Johnson <ronljohnsonjr@gmail.com> wrote:
PG 14.10

pg_ctl failed when I added a thirteenth parameter (log_temp_files).  Is this documented, and I missed it?

(The purpose is to optimize for a 30-thread pg_restore of a multi-TB 9.6 database.  Once it's finished, I stop/start, to enable "production" values like fsync=on.)

pg_ctl restart -wt999 -mfast
          -o '-c fsync=off' \
          -o '-c full_page_writes=off' \
          -o '-c log_statement=none' \
          -o '-c log_temp_files=100kB' \
          -o '-c max_wal_size=36GB' \
          -o '-c wal_level=minimal' \
          -o '-c max_wal_senders=0' \
          -o '-c wal_buffers=64MB' \
          -o '-c log_checkpoints=off' \
          -o '-c autovacuum=off'
pg_ctl: too many command-line arguments (first is "restart")

Arguments are not parameters.  Try putting the restart argument after all parameters.

Quite irritating. "pg_ctl restart" instantly tells you what it's doing.

And still, why the limitation?
 
Also, you are missing a line continuation backslash so you actually only supplied three arguments, where only 1 is accepted.
 
The missing backslash is a legacy of reformatting it for email. 

Re: pg_ctl only allows 12 parameters?

От
"David G. Johnston"
Дата:
On Mon, Dec 18, 2023 at 8:31 AM Ron Johnson <ronljohnsonjr@gmail.com> wrote:
On Mon, Dec 18, 2023 at 10:11 AM David G. Johnston <david.g.johnston@gmail.com> wrote:
On Monday, December 18, 2023, Ron Johnson <ronljohnsonjr@gmail.com> wrote:
PG 14.10

pg_ctl failed when I added a thirteenth parameter (log_temp_files).  Is this documented, and I missed it?

(The purpose is to optimize for a 30-thread pg_restore of a multi-TB 9.6 database.  Once it's finished, I stop/start, to enable "production" values like fsync=on.)

pg_ctl restart -wt999 -mfast
          -o '-c fsync=off' \
          -o '-c full_page_writes=off' \
          -o '-c log_statement=none' \
          -o '-c log_temp_files=100kB' \
          -o '-c max_wal_size=36GB' \
          -o '-c wal_level=minimal' \
          -o '-c max_wal_senders=0' \
          -o '-c wal_buffers=64MB' \
          -o '-c log_checkpoints=off' \
          -o '-c autovacuum=off'
pg_ctl: too many command-line arguments (first is "restart")

Arguments are not parameters.  Try putting the restart argument after all parameters.

Quite irritating. "pg_ctl restart" instantly tells you what it's doing.

And still, why the limitation?

I'm thinking that the forced ordering being different than what is documented is a shell-specific issue.  As for why only one of restart, stop, start, etc... can be specified for any given invocation of pg_ctl, that seems like an entirely realistic limitation and typo-preventing feature.

David J.

Re: pg_ctl only allows 12 parameters?

От
Ron Johnson
Дата:
On Mon, Dec 18, 2023 at 10:41 AM David G. Johnston <david.g.johnston@gmail.com> wrote:
On Mon, Dec 18, 2023 at 8:31 AM Ron Johnson <ronljohnsonjr@gmail.com> wrote:
On Mon, Dec 18, 2023 at 10:11 AM David G. Johnston <david.g.johnston@gmail.com> wrote:
On Monday, December 18, 2023, Ron Johnson <ronljohnsonjr@gmail.com> wrote:
PG 14.10

pg_ctl failed when I added a thirteenth parameter (log_temp_files).  Is this documented, and I missed it?

(The purpose is to optimize for a 30-thread pg_restore of a multi-TB 9.6 database.  Once it's finished, I stop/start, to enable "production" values like fsync=on.)

pg_ctl restart -wt999 -mfast
          -o '-c fsync=off' \
          -o '-c full_page_writes=off' \
          -o '-c log_statement=none' \
          -o '-c log_temp_files=100kB' \
          -o '-c max_wal_size=36GB' \
          -o '-c wal_level=minimal' \
          -o '-c max_wal_senders=0' \
          -o '-c wal_buffers=64MB' \
          -o '-c log_checkpoints=off' \
          -o '-c autovacuum=off'
pg_ctl: too many command-line arguments (first is "restart")

Arguments are not parameters.  Try putting the restart argument after all parameters.

Quite irritating. "pg_ctl restart" instantly tells you what it's doing.

And still, why the limitation?

I'm thinking that the forced ordering being different than what is documented is a shell-specific issue.  As for why only one of restart, stop, start, etc... can be specified for any given invocation of pg_ctl, that seems like an entirely realistic limitation and typo-preventing feature.

The error message is "pg_ctl: too many command-line arguments".  It's manifestly obvious that the command failed when I added a "-" option, not a "start", "stop" command. and yet pg_ctl complained about command-line arguments.

Re: pg_ctl only allows 12 parameters?

От
"David G. Johnston"
Дата:
On Mon, Dec 18, 2023 at 8:48 AM Ron Johnson <ronljohnsonjr@gmail.com> wrote:
On Mon, Dec 18, 2023 at 10:41 AM David G. Johnston <david.g.johnston@gmail.com> wrote:
On Mon, Dec 18, 2023 at 8:31 AM Ron Johnson <ronljohnsonjr@gmail.com> wrote:
On Mon, Dec 18, 2023 at 10:11 AM David G. Johnston <david.g.johnston@gmail.com> wrote:
On Monday, December 18, 2023, Ron Johnson <ronljohnsonjr@gmail.com> wrote:
PG 14.10

pg_ctl failed when I added a thirteenth parameter (log_temp_files).  Is this documented, and I missed it?

(The purpose is to optimize for a 30-thread pg_restore of a multi-TB 9.6 database.  Once it's finished, I stop/start, to enable "production" values like fsync=on.)

pg_ctl restart -wt999 -mfast
          -o '-c fsync=off' \
          -o '-c full_page_writes=off' \
          -o '-c log_statement=none' \
          -o '-c log_temp_files=100kB' \
          -o '-c max_wal_size=36GB' \
          -o '-c wal_level=minimal' \
          -o '-c max_wal_senders=0' \
          -o '-c wal_buffers=64MB' \
          -o '-c log_checkpoints=off' \
          -o '-c autovacuum=off'
pg_ctl: too many command-line arguments (first is "restart")

Arguments are not parameters.  Try putting the restart argument after all parameters.

Quite irritating. "pg_ctl restart" instantly tells you what it's doing.

And still, why the limitation?

I'm thinking that the forced ordering being different than what is documented is a shell-specific issue.  As for why only one of restart, stop, start, etc... can be specified for any given invocation of pg_ctl, that seems like an entirely realistic limitation and typo-preventing feature.

The error message is "pg_ctl: too many command-line arguments".  It's manifestly obvious that the command failed when I added a "-" option, not a "start", "stop" command. and yet pg_ctl complained about command-line arguments.

As soon as you supplied "restart" the CLI parser code for your shell/whatnot stopped looking for options/parameters and everything else subsequent is defined to be an argument.  As soon as it sees a second argument for a command that is defined to only have a single argument the parser immediately aborts.

Yes, I'm describing what is observed.  I don't know precisely which tooling is "at fault" here or who could make it behave differently but the concept of options first, everything after the last option/first argument is an argument is something that exists and is what you are seeing here.

David J.

Re: pg_ctl only allows 12 parameters?

От
Tom Lane
Дата:
Ron Johnson <ronljohnsonjr@gmail.com> writes:
> On Mon, Dec 18, 2023 at 10:41 AM David G. Johnston <
> david.g.johnston@gmail.com> wrote:
>> I'm thinking that the forced ordering being different than what is
>> documented is a shell-specific issue.  As for why only one of restart,
>> stop, start, etc... can be specified for any given invocation of pg_ctl,
>> that seems like an entirely realistic limitation and typo-preventing
>> feature.

> The error message is "pg_ctl: *too many command-line arguments*".  It's
> manifestly obvious that the command failed when I added a "-" option, not a
> "start", "stop" command. and yet pg_ctl complained about command-line
> arguments.

The program's coding expects that optional switches will appear before
the action specifier.  You can get away with writing the action first
only if the version of getopt_long() that's in use will permute the
argument array to move non-options to the end.  That's always true on
glibc-based platforms, but otherwise usually not, at least in PG v14.

(Yes, I'm aware that pg_ctl's --help message claims this works
everywhere.  It's lying.)

Having said that, the most obvious way to explain the results you
seem to be getting is that you're using a version of getopt_long()
that tries to permute the arguments and is getting it wrong.
So, what's the platform?  And are you quite sure your psql is v14?

            regards, tom lane



Re: pg_ctl only allows 12 parameters?

От
Ron Johnson
Дата:
On Mon, Dec 18, 2023 at 11:09 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
Ron Johnson <ronljohnsonjr@gmail.com> writes:
> On Mon, Dec 18, 2023 at 10:41 AM David G. Johnston <
> david.g.johnston@gmail.com> wrote:
>> I'm thinking that the forced ordering being different than what is
>> documented is a shell-specific issue.  As for why only one of restart,
>> stop, start, etc... can be specified for any given invocation of pg_ctl,
>> that seems like an entirely realistic limitation and typo-preventing
>> feature.

> The error message is "pg_ctl: *too many command-line arguments*".  It's
> manifestly obvious that the command failed when I added a "-" option, not a
> "start", "stop" command. and yet pg_ctl complained about command-line
> arguments.

The program's coding expects that optional switches will appear before
the action specifier.  You can get away with writing the action first
only if the version of getopt_long() that's in use will permute the
argument array to move non-options to the end.  That's always true on
glibc-based platforms, but otherwise usually not, at least in PG v14.

(Yes, I'm aware that pg_ctl's --help message claims this works
everywhere.  It's lying.)

Having said that, the most obvious way to explain the results you
seem to be getting is that you're using a version of getopt_long()
that tries to permute the arguments and is getting it wrong.
So, what's the platform?  And are you quite sure your psql is v14?

$ psql -V
psql (PostgreSQL) 14.10
$ postgres -V
postgres (PostgreSQL) 14.10
$ cat /etc/redhat-release
Red Hat Enterprise Linux release 8.9 (Ootpa)
# This is a "SLES Expanded Support platform 8.9"
# The above "Red Hat Enterprise Linux Server" string is only used to
# keep software compatibility.

$ yum list installed 'postgresql*'
Installed Packages
postgresql14.x86_64               14.10-1PGDG.rhel8   @@commandline
postgresql14-contrib.x86_64       14.10-1PGDG.rhel8   @@commandline
postgresql14-docs.x86_64          14.10-1PGDG.rhel8   @@commandline
postgresql14-libs.x86_64          14.10-1PGDG.rhel8   @@commandline
postgresql14-plperl.x86_64        14.10-1PGDG.rhel8   @@commandline
postgresql14-plpython3.x86_64     14.10-1PGDG.rhel8   @@commandline
postgresql14-server.x86_64        14.10-1PGDG.rhel8   @@commandline
 

Re: pg_ctl only allows 12 parameters?

От
Tom Lane
Дата:
Ron Johnson <ronljohnsonjr@gmail.com> writes:
> On Mon, Dec 18, 2023 at 11:09 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> So, what's the platform?  And are you quite sure your psql is v14?

> $ psql -V
> psql (PostgreSQL) 14.10
> $ postgres -V
> postgres (PostgreSQL) 14.10
> $ cat /etc/redhat-release
> Red Hat Enterprise Linux release 8.9 (Ootpa)
> # This is a "SLES Expanded Support platform 8.9"
> # The above "Red Hat Enterprise Linux Server" string is only used to
> # keep software compatibility.

Hmph.  I'm on (genuine) RHEL 8.9, and when I tried the command you
showed it seemed to work fine.  So there's some important detail
you've not provided.  Now I'm wondering about things like invisible
characters and whether you managed to type something that looks like
a "-" but is really some weird unicode character.

Looking at the v14 pg_ctl code, I see it has an extra level of
looping around getopt_long to try to cope with implementations
that don't permute the arguments.  So in theory it should work as
documented everywhere.  But I could see how things might get
confused, if the subroutine misidentifies what is a switch.

            regards, tom lane



Re: pg_ctl only allows 12 parameters?

От
Ron Johnson
Дата:
On Mon, Dec 18, 2023 at 12:02 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
Ron Johnson <ronljohnsonjr@gmail.com> writes:
> On Mon, Dec 18, 2023 at 11:09 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> So, what's the platform?  And are you quite sure your psql is v14?

> $ psql -V
> psql (PostgreSQL) 14.10
> $ postgres -V
> postgres (PostgreSQL) 14.10
> $ cat /etc/redhat-release
> Red Hat Enterprise Linux release 8.9 (Ootpa)
> # This is a "SLES Expanded Support platform 8.9"
> # The above "Red Hat Enterprise Linux Server" string is only used to
> # keep software compatibility.

Hmph.  I'm on (genuine) RHEL 8.9, and when I tried the command you
showed it seemed to work fine.  So there's some important detail
you've not provided.  Now I'm wondering about things like invisible
characters and whether you managed to type something that looks like
a "-" but is really some weird unicode character.

Looking at the v14 pg_ctl code, I see it has an extra level of
looping around getopt_long to try to cope with implementations
that don't permute the arguments.  So in theory it should work as
documented everywhere.  But I could see how things might get
confused, if the subroutine misidentifies what is a switch.

There must be some PEBKAC involved that pg_ctl error handling didn't explain well, because _this_ unreformatted command works. (I changed the script, to make the startup options explicit.)

pg_ctl restart -wt999 -mfast \
                -o "-c fsync=off" \
                -o "-c full_page_writes=off" \
                -o "-c log_statement=none" \
                -o "-c log_temp_files=100kB" \
                -o "-c maintenance_work_mem=2GB" \
                -o "-c max_wal_size=36GB" \
                -o "-c wal_level=minimal" \
                -o "-c max_wal_senders=0" \
                -o "-c wal_buffers=64MB" \
                -o "-c log_checkpoints=off" \
                -o "-c autovacuum=off"

$ pg_ctl status
pg_ctl: server is running (PID: 133557)
/usr/pgsql-14/bin/postgres "-c" "fsync=off" "-c" "full_page_writes=off" "-c" "log_statement=none" "-c" "log_temp_files=100kB" "-c" "maintenance_work_mem=2GB" "-c" "max_wal_size=36GB" "-c" "wal_level=minimal" "-c" "max_wal_senders=0" "-c" "wal_buffers=64MB" "-c" "log_checkpoints=off" "-c" "autovacuum=off"
 

Re: pg_ctl only allows 12 parameters?

От
Tom Lane
Дата:
Ron Johnson <ronljohnsonjr@gmail.com> writes:
> On Mon, Dec 18, 2023 at 12:02 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> Looking at the v14 pg_ctl code, I see it has an extra level of
>> looping around getopt_long to try to cope with implementations
>> that don't permute the arguments.  So in theory it should work as
>> documented everywhere.  But I could see how things might get
>> confused, if the subroutine misidentifies what is a switch.

> There must be some PEBKAC involved that pg_ctl error handling didn't
> explain well, because _this_ unreformatted command works.

After studying the code some more, I think I can reproduce it:

$ pg_ctl restart foo -o "whatever"
pg_ctl: too many command-line arguments (first is "restart")
Try "pg_ctl --help" for more information.

It should be complaining about "foo", but the message mentions
"restart" instead.  I think you must have had something in your
non-working command that did not get interpreted as being a
switch or a switch argument, leading to the misleading report.

That's with v14.  HEAD gets this right:

$ pg_ctl restart foo -o "whatever"
pg_ctl: too many command-line arguments (first is "foo")
Try "pg_ctl --help" for more information.

The change in behavior is evidently due to commit 03d1080d8, which
got rid of the extra level of looping in pg_ctl's use of getopt_long.
I surmise that after we loop around and re-invoke getopt_long,
glibc's getopt_long resets "optind" to the first non-option argument,
rather than honoring the increment from the bottom of the loop.

This seems like a bug to me, but I'm not sure if I want to
propose back-patching 03d1080d8 (and 411b72034) to deal with
it.  That code hasn't made it into a stable branch yet, so
a back-patch would be a little bit risky.

            regards, tom lane