Re: speed up a logical replica setup

On Thu, 21 Mar 2024 at 18:02, Peter Eisentraut <peter@eisentraut.org> wrote:
>
> On 21.03.24 12:35, vignesh C wrote:
> > Here are a few suggestions:
> > 1) I felt displaying the server log to the console is not a good idea,
> > I prefer this to be logged. There were similar comments from
> > Kuroda-san at [1], Peter at [2]. The number of lines will increase
> > based on the log level set. If you don't want to use pg_upgrade style,
> > how about exposing the log file option and logging it to the specified
> > log file.
>
> Let's leave that for the next version.  We need to wrap things up for
> this release.

Ok, This can be done as an improvement.

> > 2) Currently for publication, replication-slot and subscription, we
> > will have to specify these options based on the number of databases.
> > Here if we have 100 databases we will have to specify these options
> > 100 times, it might not be user friendly. How about something like
> > what Tomas had proposed at [3] and  Amit proposed at [4]. It will be
> > better if the user has to just specify publication, replication slot
> > and subscription options only one time.
>
> Same.  Designing, implementing, discussing, and testing this cannot be
> done in the time remaining.

If we commit this we might not be able to change the way the option
behaves once the customers starts using it. How about removing these
options in the first version and adding it in the next version after
more discussion.

> > 3) Can we have an option similar to dry-run which will display the
> > configurations required in the primary and standby node something
> > like:
> > pg_createsubscriber -D data_N2/ -P "port=5431 user=postgres"  -p 9999
> > -s /home/vignesh/postgres/inst/bin/ -U postgres -d db1 -d db2
> > --suggest-config
> > Suggested optimal configurations in the primary:
> > --------------------------------------
> > wallevel = logical
> > max_replication_slots = 3
> > max_wal_senders = 3
> > ...
> > Suggested optimal configurations in the standby:
> > --------------------------------------
> > max_replication_slots = 3
> > max_wal_senders = 3
> > ...
>
> How would this be different from what --dry-run does now?

Currently dry-run will do the check and might fail on identifying a
few failures like after checking subscriber configurations. Then the
user will have to correct the configuration and re-run then fix the
next set of failures. Whereas the suggest-config will display all the
optimal configuration for both the primary and the standby in a single
shot. This is not a must in the first version, it can be done as a
subsequent enhancement.

Regards,
Vignesh