Обсуждение: pgsql: Add decoding of sequences to test_decoding

Поиск
Список
Период
Сортировка

pgsql: Add decoding of sequences to test_decoding

От
Tomas Vondra
Дата:
Add decoding of sequences to test_decoding

Commit 0da92dc530 improved the logical decoding infrastructure to handle
sequences, and did various changes to related parts (WAL logging etc.).
But it did not include any implementation of the new callbacks added to
OutputPluginCallbacks.

This extends test_decoding with two callbacks to decode sequences. The
decoding of sequences may be disabled using 'include-sequences', a new
option of the output plugin.

Author: Tomas Vondra, Cary Huang
Reviewed-by: Peter Eisentraut, Hannu Krosing, Andres Freund
Discussion: https://postgr.es/m/d045f3c2-6cfb-06d3-5540-e63c320df8bc@enterprisedb.com
Discussion: https://postgr.es/m/1710ed7e13b.cd7177461430746.3372264562543607781@highgo.ca

Branch
------
master

Details
-------
https://git.postgresql.org/pg/commitdiff/80901b32913ffa59bf157a4d88284b2b3a7511d9

Modified Files
--------------
contrib/test_decoding/Makefile                     |   3 +-
contrib/test_decoding/expected/ddl.out             |  12 +-
.../test_decoding/expected/decoding_in_xact.out    |   2 +-
.../test_decoding/expected/decoding_into_rel.out   |  10 +-
contrib/test_decoding/expected/mxact.out           |   8 +-
contrib/test_decoding/expected/ondisk_startup.out  |   4 +-
contrib/test_decoding/expected/replorigin.out      |   4 +-
contrib/test_decoding/expected/rewrite.out         |   4 +-
contrib/test_decoding/expected/sequence.out        | 327 +++++++++++++++++++++
contrib/test_decoding/expected/slot.out            |   2 +-
contrib/test_decoding/expected/toast.out           |  10 +-
contrib/test_decoding/expected/truncate.out        |   2 +-
contrib/test_decoding/specs/mxact.spec             |   2 +-
contrib/test_decoding/specs/ondisk_startup.spec    |   2 +-
contrib/test_decoding/sql/ddl.sql                  |  12 +-
contrib/test_decoding/sql/decoding_in_xact.sql     |   2 +-
contrib/test_decoding/sql/decoding_into_rel.sql    |  10 +-
contrib/test_decoding/sql/replorigin.sql           |   4 +-
contrib/test_decoding/sql/rewrite.sql              |   4 +-
contrib/test_decoding/sql/sequence.sql             | 119 ++++++++
contrib/test_decoding/sql/slot.sql                 |   2 +-
contrib/test_decoding/sql/toast.sql                |  10 +-
contrib/test_decoding/sql/truncate.sql             |   2 +-
contrib/test_decoding/test_decoding.c              |  65 ++++
24 files changed, 567 insertions(+), 55 deletions(-)


Re: pgsql: Add decoding of sequences to test_decoding

От
Andres Freund
Дата:
Hi,

On 2022-02-11 23:52:20 +0000, Tomas Vondra wrote:
> Add decoding of sequences to test_decoding
>
> Commit 0da92dc530 improved the logical decoding infrastructure to handle
> sequences, and did various changes to related parts (WAL logging etc.).
> But it did not include any implementation of the new callbacks added to
> OutputPluginCallbacks.
>
> This extends test_decoding with two callbacks to decode sequences. The
> decoding of sequences may be disabled using 'include-sequences', a new
> option of the output plugin.
>
> Author: Tomas Vondra, Cary Huang
> Reviewed-by: Peter Eisentraut, Hannu Krosing, Andres Freund
> Discussion: https://postgr.es/m/d045f3c2-6cfb-06d3-5540-e63c320df8bc@enterprisedb.com
> Discussion: https://postgr.es/m/1710ed7e13b.cd7177461430746.3372264562543607781@highgo.ca

The tests fail on slow machines (valgrind in this case), with some spurious
output differences:

https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=skink&dt=2022-02-12%2010%3A24%3A22

diff -U3 /mnt/resource/bf/build/skink-master/HEAD/pgsql/contrib/test_decoding/expected/sequence.out
/mnt/resource/bf/build/skink-master/HEAD/pgsql.build/contrib/test_decoding/results/sequence.out
--- /mnt/resource/bf/build/skink-master/HEAD/pgsql/contrib/test_decoding/expected/sequence.out    2022-02-12
00:40:25.431913374+0000
 
+++ /mnt/resource/bf/build/skink-master/HEAD/pgsql.build/contrib/test_decoding/results/sequence.out    2022-02-12
11:07:19.177487256+0000
 
@@ -263,12 +263,14 @@
                                           data
 -----------------------------------------------------------------------------------------
  BEGIN
+ COMMIT
+ BEGIN
  sequence public.test_table_a_seq: transactional:1 last_value: 1 log_cnt: 0 is_called:0
  sequence public.test_table_a_seq: transactional:1 last_value: 33 log_cnt: 0 is_called:1
  table public.test_table: INSERT: a[integer]:1 b[integer]:100
  table public.test_table: INSERT: a[integer]:2 b[integer]:200
  COMMIT
-(6 rows)
+(8 rows)

I don't think it's a good idea to use skip-empty-xacts = 0 in tests, there
always can be an autovacuum or such.

Greetings,

Andres Freund



Re: pgsql: Add decoding of sequences to test_decoding

От
Tomas Vondra
Дата:
On 2/12/22 23:04, Andres Freund wrote:
> Hi,
> 
> On 2022-02-11 23:52:20 +0000, Tomas Vondra wrote:
>> Add decoding of sequences to test_decoding
>>
>> Commit 0da92dc530 improved the logical decoding infrastructure to handle
>> sequences, and did various changes to related parts (WAL logging etc.).
>> But it did not include any implementation of the new callbacks added to
>> OutputPluginCallbacks.
>>
>> This extends test_decoding with two callbacks to decode sequences. The
>> decoding of sequences may be disabled using 'include-sequences', a new
>> option of the output plugin.
>>
>> Author: Tomas Vondra, Cary Huang
>> Reviewed-by: Peter Eisentraut, Hannu Krosing, Andres Freund
>> Discussion: https://postgr.es/m/d045f3c2-6cfb-06d3-5540-e63c320df8bc@enterprisedb.com
>> Discussion: https://postgr.es/m/1710ed7e13b.cd7177461430746.3372264562543607781@highgo.ca
> 
> The tests fail on slow machines (valgrind in this case), with some spurious
> output differences:
> 
> https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=skink&dt=2022-02-12%2010%3A24%3A22
> 
> diff -U3 /mnt/resource/bf/build/skink-master/HEAD/pgsql/contrib/test_decoding/expected/sequence.out
/mnt/resource/bf/build/skink-master/HEAD/pgsql.build/contrib/test_decoding/results/sequence.out
> --- /mnt/resource/bf/build/skink-master/HEAD/pgsql/contrib/test_decoding/expected/sequence.out    2022-02-12
00:40:25.431913374+0000
 
> +++ /mnt/resource/bf/build/skink-master/HEAD/pgsql.build/contrib/test_decoding/results/sequence.out    2022-02-12
11:07:19.177487256+0000
 
> @@ -263,12 +263,14 @@
>                                            data
>  -----------------------------------------------------------------------------------------
>   BEGIN
> + COMMIT
> + BEGIN
>   sequence public.test_table_a_seq: transactional:1 last_value: 1 log_cnt: 0 is_called:0
>   sequence public.test_table_a_seq: transactional:1 last_value: 33 log_cnt: 0 is_called:1
>   table public.test_table: INSERT: a[integer]:1 b[integer]:100
>   table public.test_table: INSERT: a[integer]:2 b[integer]:200
>   COMMIT
> -(6 rows)
> +(8 rows)
> 
> I don't think it's a good idea to use skip-empty-xacts = 0 in tests, there
> always can be an autovacuum or such.
> 

Thanks, I pushed a fix. Turns out the bug was a bit worse, because the
sequence callbacks did not actually deal with empty xacts correctly, so
I fixed that too.


regards

-- 
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company