Обсуждение: Issues with PG15beta3

Поиск
Список
Период
Сортировка

Issues with PG15beta3

От
Didier Michel
Дата:
Hello,

I am integrating new pg_basebackup zstd compression functionality in Barman for compressed backup.

I do encounter two main issues when using zstd compression algorithm:

On ubuntu 18.04 : 
- command exemple:  /usr/bin/pg_basebackup --dbname=dbname=replication host=db options=-cdatestyle=iso port=5432 replication=true user=barman_streaming application_name=barman_streaming_backup -v --no-password
--pgdata=/var/lib/barman/main/base/20220811T162557/data --no-slot --wal-method=none
--checkpoint=fast 
 --format=tar
--compress=zstd:level=5

Interesting part is "--compress=zstd:level=5"
- pg_basebackup response: 
pg_basebackup: checkpoint completed
pg_basebackup: error: this build does not support zstd compression


On Ubuntu 20.04 and 22.04:
The same command works. But When I try to introduce workers option (ie "--compress=zstd:level=5,workers=2") it fails with following error:
pg_basebackup: checkpoint completed
pg_basebackup: error: could not set compression worker count to 2: Unsupported parameter

Unless I have misused pg_basebackup (which is completely possible), I am wondering if there could be an issue with PG packaging  or compilation (especially for workers that triggers threads)

Hoping this can help deliver PG15.
Regards.

Didier Michel

Re: Issues with PG15beta3

От
Christoph Berg
Дата:
Re: Didier Michel
> On ubuntu 18.04 :
> pg_basebackup: error: this build does not support zstd compression

Hi Didier,

bionic's zstd is too old, PG requires at least 1.4.0.

libzstd-dev | 1.3.3+dfsg-2ubuntu1   | bionic          | amd64, arm64, armhf, i386, ppc64el, s390x

Christoph



Re: Issues with PG15beta3

От
Didier Michel
Дата:
Ho thank you Christoph I only checked pg_basebackup version   

Didier

On Fri, Aug 12, 2022 at 7:38 PM Christoph Berg <myon@debian.org> wrote:
Re: Didier Michel
> On ubuntu 18.04 :
> pg_basebackup: error: this build does not support zstd compression

Hi Didier,

bionic's zstd is too old, PG requires at least 1.4.0.

libzstd-dev | 1.3.3+dfsg-2ubuntu1   | bionic          | amd64, arm64, armhf, i386, ppc64el, s390x

Christoph

Re: Issues with PG15beta3

От
Didier Michel
Дата:
Do you have any ideas regarding ubuntu focal and Jammy ?

Didier


Focal?


On Sat, Aug 13, 2022 at 1:26 PM Didier Michel <didier.michel@enterprisedb.com> wrote:
Ho thank you Christoph I only checked pg_basebackup version   

Didier

On Fri, Aug 12, 2022 at 7:38 PM Christoph Berg <myon@debian.org> wrote:
Re: Didier Michel
> On ubuntu 18.04 :
> pg_basebackup: error: this build does not support zstd compression

Hi Didier,

bionic's zstd is too old, PG requires at least 1.4.0.

libzstd-dev | 1.3.3+dfsg-2ubuntu1   | bionic          | amd64, arm64, armhf, i386, ppc64el, s390x

Christoph

Re: Issues with PG15beta3

От
Christoph Berg
Дата:
Re: Didier Michel
> Do you have any ideas regarding ubuntu focal and Jammy ?

What ideas?

Christoph



Re: Issues with PG15beta3

От
Didier Michel
Дата:
Sorry, I did not include previous message:
With focal and Jammy, pg_basebackup returns another error when I try to use several workers (ie "--compress=zstd:level=5,workers=2")

pg_basebackup: checkpoint completed
pg_basebackup: error: could not set compression worker count to 2: Unsupported parameter

I checked zstd version and it is higher then 1.4.0 (1.4.4). So I am wondering if there is something else I should check.

Regards



On Tue, Aug 23, 2022 at 10:10 AM Christoph Berg <myon@debian.org> wrote:
Re: Didier Michel
> Do you have any ideas regarding ubuntu focal and Jammy ?

What ideas?

Christoph

Re: Issues with PG15beta3

От
Christoph Berg
Дата:
Re: Didier Michel
> Sorry, I did not include previous message:
> With focal and Jammy, pg_basebackup returns another error when I try to use
> several workers (ie "--compress=zstd:level=5,workers=2")
> 
> pg_basebackup: checkpoint completed
> pg_basebackup: error: could not set compression worker count to 2:
> Unsupported parameter

Try without the workers part? I guess that needs a yet newer zstd
version.

Christoph



Re: Issues with PG15beta3

От
Didier Michel
Дата:
Thank you for your feedback.

I found this on zstd code that also help a lot:
On older versions of libzstd, this option does not exist, and
trying to set it will fail. Similarly for newer versions if they
are compiled without threading support

So I will check (not sure yet how ) if zstd packaged in ubuntu Jammy is compiled with threading support.

Thank you again for your help and time.
Didier


On Tue, Aug 23, 2022 at 11:06 AM Christoph Berg <myon@debian.org> wrote:
Re: Didier Michel
> Sorry, I did not include previous message:
> With focal and Jammy, pg_basebackup returns another error when I try to use
> several workers (ie "--compress=zstd:level=5,workers=2")
>
> pg_basebackup: checkpoint completed
> pg_basebackup: error: could not set compression worker count to 2:
> Unsupported parameter

Try without the workers part? I guess that needs a yet newer zstd
version.

Christoph