Re: Missing rows after migrating from postgres 11 to 12 with logical replication

Поиск
Список
Период
Сортировка
От Adrian Klaver
Тема Re: Missing rows after migrating from postgres 11 to 12 with logical replication
Дата
Msg-id c28075be-b018-3944-4ecb-31ac5b8e2ce4@aklaver.com
обсуждение исходный текст
Ответ на Re: Missing rows after migrating from postgres 11 to 12 with logical replication  (Lars Vonk <lars.vonk@gmail.com>)
Ответы Re: Missing rows after migrating from postgres 11 to 12 with logical replication  (Lars Vonk <lars.vonk@gmail.com>)
Список pgsql-general
On 12/22/20 9:10 AM, Lars Vonk wrote:
>     Did you have some other replication running on the 11 instance?
> 
> 
> Yes the 11 instance also had another (11) replica running. (But these 
> logs are from the 12 instance)

The 11 instance had the data that went missing in the 12 instance, so 
what shows up in logs for the 11 instance during this period that is 
relevant?

> 
> The new 12 instance also had a replica running.

So the setup was?:

1) 11 primary --> 11 standby via what replication logical or binary?
     | -->         12 new instance via logical

2) 12(new) primary --> 12(?) standby via what replication logical or binary?

> 
>     In any case what was the command logged just before the ERROR.
> 
> 
> There is nothing logged.
> 
> These are the only log statements just before the error message, one 
> second later the ERROR is logged:
> 
> 2020-12-10 13:26:43 UTC::@:[5537]:LOG:  checkpoints are occurring too 
> frequently (20 seconds apart)
> 2020-12-10 13:26:43 UTC::@:[5537]:HINT:  Consider increasing the 
> configuration parameter "max_wal_size".
> 2020-12-10 13:26:43 UTC::@:[5537]:LOG:  checkpoint starting: wal
> 
> Lars
> 
> On Mon, Dec 21, 2020 at 11:51 PM Adrian Klaver 
> <adrian.klaver@aklaver.com <mailto:adrian.klaver@aklaver.com>> wrote:
> 
>     On 12/21/20 2:42 PM, Lars Vonk wrote:
>      >     What was being run when the above ERROR was triggered?
>      >
>      >
>      > The initial copy of a table. Other than that we ran select
>      > pg_size_pretty(pg_relation_size('table_name')) to see the current
>     size
>      > of the table being copied to get a feeling on progress.
>      >
>      > And whenever we added a new table to the publication we ran ALTER
>      > SUBSCRIPTION migration REFRESH PUBLICATION; to add any new table
>     to the
>      > subscription. But not around that timestamp, about 50 minutes
>     before the
>      > first occurence of that ERROR. (no ERRORS after prior ALTER
>     SUBSCRIPTIONs).
>      >
>      > But after the initial copy's ended there are more ERROR's on
>     different
>      > WAL segments missing. Each missing wal segment is logged as ERROR a
>      > couple of times and then no more. After a couple of hours no
>     errors are
>      > logged.
> 
>     Something was looking for the WAL segment.
> 
>     Did you have some other replication running on the 11 instance?
> 
>     In any case what was the command logged just before the ERROR.
> 
>      >
>      > Lars
>      >
> 
> 
>     -- 
>     Adrian Klaver
>     adrian.klaver@aklaver.com <mailto:adrian.klaver@aklaver.com>
> 


-- 
Adrian Klaver
adrian.klaver@aklaver.com



В списке pgsql-general по дате отправления:

Предыдущее
От: "Mark Felder"
Дата:
Сообщение: ts_parse reports different between MacOS, FreeBSD/Linux
Следующее
От: Tom Lane
Дата:
Сообщение: Re: ts_parse reports different between MacOS, FreeBSD/Linux