Обсуждение: Postgresql Database cant start after restore using pgBackRest

Поиск
Список
Период
Сортировка

Postgresql Database cant start after restore using pgBackRest

От
Deo Felix
Дата:

 I am doing backup with pgbackrest in the database server every day at night, I was testing to restore the backup in another server, the restore was successfully however the database cant start

pgbackrest --stanza=data --delta --log-level-console=detail restoreimage.png


image.png


Вложения

Re: Postgresql Database cant start after restore using pgBackRest

От
Doug Reynolds
Дата:
Is this a case of not removing the archive command from the postgresql.conf?  Seems that the initial start of the server before restore created some empty Wal logs.

Sent from my iPhone

On Jun 25, 2023, at 12:55 PM, Deo Felix <deofelix9@gmail.com> wrote:



 I am doing backup with pgbackrest in the database server every day at night, I was testing to restore the backup in another server, the restore was successfully however the database cant start

pgbackrest --stanza=data --delta --log-level-console=detail restore

<image.png>


<image.png>


Re: Postgresql Database cant start after restore using pgBackRest

От
Ron
Дата:
On 6/25/23 11:53, Deo Felix wrote:

 I am doing backup with pgbackrest in the database server every day at night, I was testing to restore the backup in another server, the restore was successfully however the database cant start

pgbackrest --stanza=data --delta --log-level-console=detail restore


Here's what worked for me:
SetName=20221106-000003F
pgbackrest restore \
    --stanza=$SomeStanza \
    --pg1-path=$NewPGData \
    --log-level-file=detail \
    --log-level-console=detail \
    --process-max=$ThreadCnt \
    --type=immediate --set=$SetName

An alternative which should also work:
RestoreTime="2022-11-06 02:00:00.0000"
pgbackrest restore \
    --stanza=$SomeStanza \
    --pg1-path=$NewPGData \
    --log-level-file=detail \
    --log-level-console=detail \
    --process-max=$ThreadCnt \
    --type=time --target="$RestoreTime"

image.png


image.png


In Thunderbird, at least, it's impossible to see those tiny images.


--
Born in Arizona, moved to Babylonia.
Вложения

Re: Postgresql Database cant start after restore using pgBackRest

От
Deo Felix
Дата:
I have tried removing the archive command but still does not work

On Sun, 25 Jun 2023 at 20:30, Ron <ronljohnsonjr@gmail.com> wrote:
On 6/25/23 11:53, Deo Felix wrote:

 I am doing backup with pgbackrest in the database server every day at night, I was testing to restore the backup in another server, the restore was successfully however the database cant start

pgbackrest --stanza=data --delta --log-level-console=detail restore


Here's what worked for me:
SetName=20221106-000003F
pgbackrest restore \
    --stanza=$SomeStanza \
    --pg1-path=$NewPGData \
    --log-level-file=detail \
    --log-level-console=detail \
    --process-max=$ThreadCnt \
    --type=immediate --set=$SetName

An alternative which should also work:
RestoreTime="2022-11-06 02:00:00.0000"
pgbackrest restore \
    --stanza=$SomeStanza \
    --pg1-path=$NewPGData \
    --log-level-file=detail \
    --log-level-console=detail \
    --process-max=$ThreadCnt \
    --type=time --target="$RestoreTime"

image.png


image.png


In Thunderbird, at least, it's impossible to see those tiny images.


--
Born in Arizona, moved to Babylonia.
Вложения

Re: Postgresql Database cant start after restore using pgBackRest

От
Deo Felix
Дата:
I have tried removing the archive command but still does not work

On Sun, 25 Jun 2023 at 20:08, Doug Reynolds <mav@wastegate.net> wrote:
Is this a case of not removing the archive command from the postgresql.conf?  Seems that the initial start of the server before restore created some empty Wal logs.

Sent from my iPhone

On Jun 25, 2023, at 12:55 PM, Deo Felix <deofelix9@gmail.com> wrote:



 I am doing backup with pgbackrest in the database server every day at night, I was testing to restore the backup in another server, the restore was successfully however the database cant start

pgbackrest --stanza=data --delta --log-level-console=detail restore

<image.png>


<image.png>


Re: Postgresql Database cant start after restore using pgBackRest

От
Deo Felix
Дата:
image.png


<2023-06-25 21:14:17.647 EAT::@:[1038849]:>LOG:  restored log file "0000000100001834000000F4" from archive
<2023-06-25 21:14:18.311 EAT::@:[1038841]:>LOG:  received smart shutdown request
<2023-06-25 21:14:18.386 EAT::@:[1038841]:>LOG:  received SIGHUP, reloading configuration files
<2023-06-25 21:14:18.507 EAT::@:[1038841]:>LOG:  startup process (PID 1038849) was terminated by signal 9: Killed
<2023-06-25 21:14:18.507 EAT::@:[1038841]:>LOG:  terminating any other active server processes
<2023-06-25 21:14:18.507 EAT::@:[1038841]:>LOG:  abnormal database system shutdown
<2023-06-25 21:14:19.438 EAT::@:[1038841]:>LOG:  database system is shut down

On Sun, 25 Jun 2023 at 20:30, Ron <ronljohnsonjr@gmail.com> wrote:
On 6/25/23 11:53, Deo Felix wrote:

 I am doing backup with pgbackrest in the database server every day at night, I was testing to restore the backup in another server, the restore was successfully however the database cant start

pgbackrest --stanza=data --delta --log-level-console=detail restore


Here's what worked for me:
SetName=20221106-000003F
pgbackrest restore \
    --stanza=$SomeStanza \
    --pg1-path=$NewPGData \
    --log-level-file=detail \
    --log-level-console=detail \
    --process-max=$ThreadCnt \
    --type=immediate --set=$SetName

An alternative which should also work:
RestoreTime="2022-11-06 02:00:00.0000"
pgbackrest restore \
    --stanza=$SomeStanza \
    --pg1-path=$NewPGData \
    --log-level-file=detail \
    --log-level-console=detail \
    --process-max=$ThreadCnt \
    --type=time --target="$RestoreTime"

image.png


image.png


In Thunderbird, at least, it's impossible to see those tiny images.


--
Born in Arizona, moved to Babylonia.
Вложения

Re: Postgresql Database cant start after restore using pgBackRest

От
David Steele
Дата:
On 6/25/23 18:53, Deo Felix wrote:
>   I am doing backup with pgbackrest in the database server every day at 
> night, I was testing to restore the backup in another server, the 
> restore was successfully however the database cant start
> 
> *pgbackrest --stanza=data --delta --log-level-console=detail restore*

Usually this happens when you promote a standby while the primary is 
still running. The promotion of the standby will create a new timeline 
(in this case 2) and then push WAL to the archive on this new timeline.

When you restore, PostgreSQL will try to recover to timeline 2 (by 
default) even though timeline 2 can't be reached from timeline 1 backups 
that were made after the promotion.

So, you need to instruct PostgreSQL to stay on timeline 1 rather than 
try to switch to timeline 2. You can do that by setting 
--target-timeline=current, i.e. stay on the timeline that was current 
when the backup was made.

Regards,
-David



Re: Postgresql Database cant start after restore using pgBackRest

От
Ron
Дата:
On 6/26/23 08:08, David Steele wrote:
> On 6/25/23 18:53, Deo Felix wrote:
>>   I am doing backup with pgbackrest in the database server every day at 
>> night, I was testing to restore the backup in another server, the restore 
>> was successfully however the database cant start
>>
>> *pgbackrest --stanza=data --delta --log-level-console=detail restore*
>
> Usually this happens when you promote a standby while the primary is still 
> running. The promotion of the standby will create a new timeline (in this 
> case 2) and then push WAL to the archive on this new timeline.
>
> When you restore, PostgreSQL will try to recover to timeline 2 (by 
> default) even though timeline 2 can't be reached from timeline 1 backups 
> that were made after the promotion.
>
> So, you need to instruct PostgreSQL to stay on timeline 1 rather than try 
> to switch to timeline 2. You can do that by setting 
> --target-timeline=current, i.e. stay on the timeline that was current when 
> the backup was made.

Did I not have this problem because I restored to a different pg-path?

-- 
Born in Arizona, moved to Babylonia.



Re: Postgresql Database cant start after restore using pgBackRest

От
Deo Felix
Дата:
In the postgresql.auto.conf i have this configs
restore_command = 'pgbackrest --pg1-path=/var/lib/postgres/main --stanza=data archive-get %f "%p"'
recovery_target = 'immediate'
recovery_target_timeline = 'current'

But still I the database cant start


image.png

On Mon, 26 Jun 2023 at 16:55, Ron <ronljohnsonjr@gmail.com> wrote:
On 6/26/23 08:08, David Steele wrote:
> On 6/25/23 18:53, Deo Felix wrote:
>>   I am doing backup with pgbackrest in the database server every day at
>> night, I was testing to restore the backup in another server, the restore
>> was successfully however the database cant start
>>
>> *pgbackrest --stanza=data --delta --log-level-console=detail restore*
>
> Usually this happens when you promote a standby while the primary is still
> running. The promotion of the standby will create a new timeline (in this
> case 2) and then push WAL to the archive on this new timeline.
>
> When you restore, PostgreSQL will try to recover to timeline 2 (by
> default) even though timeline 2 can't be reached from timeline 1 backups
> that were made after the promotion.
>
> So, you need to instruct PostgreSQL to stay on timeline 1 rather than try
> to switch to timeline 2. You can do that by setting
> --target-timeline=current, i.e. stay on the timeline that was current when
> the backup was made.

Did I not have this problem because I restored to a different pg-path?

--
Born in Arizona, moved to Babylonia.


Вложения

Re: Postgresql Database cant start after restore using pgBackRest

От
David Steele
Дата:
On 6/26/23 15:55, Ron wrote:
> On 6/26/23 08:08, David Steele wrote:
>> On 6/25/23 18:53, Deo Felix wrote:
>>>   I am doing backup with pgbackrest in the database server every day 
>>> at night, I was testing to restore the backup in another server, the 
>>> restore was successfully however the database cant start
>>>
>>> *pgbackrest --stanza=data --delta --log-level-console=detail restore*
>>
>> Usually this happens when you promote a standby while the primary is 
>> still running. The promotion of the standby will create a new timeline 
>> (in this case 2) and then push WAL to the archive on this new timeline.
>>
>> When you restore, PostgreSQL will try to recover to timeline 2 (by 
>> default) even though timeline 2 can't be reached from timeline 1 
>> backups that were made after the promotion.
>>
>> So, you need to instruct PostgreSQL to stay on timeline 1 rather than 
>> try to switch to timeline 2. You can do that by setting 
>> --target-timeline=current, i.e. stay on the timeline that was current 
>> when the backup was made.
> 
> Did I not have this problem because I restored to a different pg-path?

I believe you did not have this problem because you do not have an 
00000002.history file in your archive to cause a problem for your recovery.

Regards,
-David



Re: Postgresql Database cant start after restore using pgBackRest

От
David Steele
Дата:
On 6/27/23 07:41, Deo Felix wrote:
> In the postgresql.auto.conf i have this configs
> restore_command = 'pgbackrest --pg1-path=/var/lib/postgres/main 
> --stanza=data archive-get %f "%p"'
> recovery_target = 'immediate'
> recovery_target_timeline = 'current'
> 
> But still I the database cant start

You will need to redo your pgbackrest restore with 
--target-timeline=current and then restart postgres to start recovery 
with the correct settings.

Regards,
-David



Re: Postgresql Database cant start after restore using pgBackRest

От
Ron
Дата:
On 6/27/23 03:26, David Steele wrote:
> On 6/27/23 07:41, Deo Felix wrote:
>> In the postgresql.auto.conf i have this configs
>> restore_command = 'pgbackrest --pg1-path=/var/lib/postgres/main 
>> --stanza=data archive-get %f "%p"'
>> recovery_target = 'immediate'
>> recovery_target_timeline = 'current'
>>
>> But still I the database cant start
>
> You will need to redo your pgbackrest restore with 
> --target-timeline=current and then restart postgres to start recovery with 
> the correct settings.

Is this standard procedure for an "in place" --delta restore?

-- 
Born in Arizona, moved to Babylonia.



Re: Postgresql Database cant start after restore using pgBackRest

От
David Steele
Дата:
On 6/27/23 12:12, Ron wrote:
> On 6/27/23 03:26, David Steele wrote:
>> On 6/27/23 07:41, Deo Felix wrote:
>>> In the postgresql.auto.conf i have this configs
>>> restore_command = 'pgbackrest --pg1-path=/var/lib/postgres/main 
>>> --stanza=data archive-get %f "%p"'
>>> recovery_target = 'immediate'
>>> recovery_target_timeline = 'current'
>>>
>>> But still I the database cant start
>>
>> You will need to redo your pgbackrest restore with 
>> --target-timeline=current and then restart postgres to start recovery 
>> with the correct settings.
> 
> Is this standard procedure for an "in place" --delta restore?

No -- this is mitigation for the split-brain situation which was causing 
the OP's recovery error.

Regards,
-David



Re: Postgresql Database cant start after restore using pgBackRest

От
Deo Felix
Дата:
Thank you, this worked 

On Tue, 27 Jun 2023 at 11:26, David Steele <david@pgmasters.net> wrote:
On 6/27/23 07:41, Deo Felix wrote:
> In the postgresql.auto.conf i have this configs
> restore_command = 'pgbackrest --pg1-path=/var/lib/postgres/main
> --stanza=data archive-get %f "%p"'
> recovery_target = 'immediate'
> recovery_target_timeline = 'current'
>
> But still I the database cant start

You will need to redo your pgbackrest restore with
--target-timeline=current and then restart postgres to start recovery
with the correct settings.

Regards,
-David