Обсуждение: Proposal: Early providing of PGDG repositories for the major Linux distributions like Fedora or Debian

Поиск
Список
Период
Сортировка

Proposal: Early providing of PGDG repositories for the major Linux distributions like Fedora or Debian

От
Hans Buschmann
Дата:

Yesterday Fedora 40 was released as GA available release.


According to the project website (download for Linux/Fedora):


" the PostgreSQL project provides a repository of packages of all supported versions for the most common distributions."

Today (24.4.2024) I upgraded my laptop to Fedora 40, but there where no repository available, so I ended with a mix of Fedora 40 and Fedora 39 installation.

To mitigate this situation, I propose to provide these repositories much earlier in the beta phase of the distributions:

Fedora easily switches from a beta to a full installation by simply dnf upgrading after GA release. So it should be possible, to test the standard repositories and installations in the beta phase.
This also could give a broader coverage of the new standard tools utilized (e.g. gcc compiler), wich changed in the current case from major version 13 to 14.

Instead of following every beta introduction of linux distros I propose to provide the repositories with (or shortly after) the standard dates for minor releases, especially in February and August of the year, wich seems well fitting the rythm of distribution releases.

The website for download also should contain the new upcoming distributions to avoid the current 404 error during installation/upgrade.

Should any serious compiler problems arise, the publication can be delayed until they are solved.

This proposal also follows the idea of getting reproducable builds and integrating build scripts into the source tree. [1]

I think there should be not much extra work, since the same minor versions of the Postgres repositories have to be build anyway at release time of the linux distribution.

I think for easy installation it is important to have a standard installation experience for all potential users so that manual installation or self compilation can be avoided.

To complete this message I would kindly ask if somebody has taken a look on bug report # 18339 which addressed another problem with fedora installations. [2]

(CC Devrim according to the thread Security lessons from libzlma - libsystemd)

regards

Hans Buschmann 


Re: Proposal: Early providing of PGDG repositories for the major Linux distributions like Fedora or Debian

От
Devrim Gündüz
Дата:
Hi,

On Wed, 2024-04-24 at 11:02 +0000, Hans Buschmann wrote:
> Today (24.4.2024) I upgraded my laptop to Fedora 40, but there where
> no repository available, so I ended with a mix of Fedora 40 and Fedora
> 39 installation.

This was caused by an unexpected and very long network outage that
blocked my access to the build instances. I released packages last
Thursday (2 days after F40 was released, which is 26 April 2024)

I sent emails to the RPM mailing list about the issue:

https://www.postgresql.org/message-id/aec36aec623741ae314692b318c890c646498ca6.camel%40gunduz.org
https://www.postgresql.org/message-id/1fe99b0def5d7539939421fa5b35db2c8f2a40ad.camel%40gunduz.org
https://www.postgresql.org/message-id/3a1b0f58673d35fae9979ed2b149972195c7b8bc.camel%40gunduz.org

> To mitigate this situation, I propose to provide these repositories
> much earlier in the beta phase of the distributions:

This is what I do for the Fedora releases. I'm sure you've noticed that
in the past.

Regards,

--
Devrim Gündüz
Open Source Solution Architect, PostgreSQL Major Contributor
Twitter: @DevrimGunduz , @DevrimGunduzTR

AW: Proposal: Early providing of PGDG repositories for the major Linux distributions like Fedora or Debian

От
Hans Buschmann
Дата:

Hello Devrim,


Sorry for the long network outage!
>>This is what I do for the Fedora releases. I'm sure you've noticed that
>>in the past.


No, I didn't notice it and I missed it recently, so I made this proposal (see 1. below).

Here are some points concerning $topic:

  1. Availability in beta phase of OS distribution 
I recently installed a new server with Fedora 39 for production and another virtual machine for test/developing with Fedora 40beta to avoid the immediatly coming system upgrade from F39 to F40.

By using the standard installation procedure from the PGDG download site I wanted to download the repository for F40 for postgres, but this was not available:

sudo dnf install https://download.postgresql.org/pub/repos/yum/reporpms/F-40-x86_64/pgdg-fedora-repo-latest.noarch.rpm

As this is the start point of the standard installation I could not install the newest release from PGDG.

Because the beta software for Fedora silently upgrades to the final version when it becomes GA, this kind of installation guaranties no interruption later on.

Early installation helps to test the installation and possible changes concerning glibc, icu, jit, compiler or similar cases.

For the simplicity I propose to make the repositories available at the last regular minor version update before the normal beta phase of the operating system starts.
This frees you from hurriying on OS GA day to provide them at time. It is highly probable that in this period neither postgres nor the OS get significant changes so every user (being aware of a beta release of an OS) is quite safe.

The schedule for many OS (I only have experience with Fedora, but similar for ubuntu, debian and certainly others) is spring time or automn time.

So the solution is:

Provide the latest minor version packages at the regulary february and august dates for beta versions of upcoming OS versions and also advertise them on the website.

2. Fast propagation of minor versions to OS distributions

I really encourage you in emphasizing downstream to use always the latest minor releases of postgres with a new OS version!

Apart from possible security fixes I see a lot of complaints on general/hackers where users report problems with outdated versions.
Perhaps upstream is not aware of the fact that minor versions are only maintenance releases and users are urged to always use the newest version. (see the corresponding discussion on hackers)

I recently saw a chart (don't remember where) of the delays major distributions have for integrating minor versions. this mostly took over 100 days, sometimes more then 200!

looking at  the example of Fedora 40 they provide PG 16.1 available on their repositories (not PGDG), more then 2 months after release of 16.2.
In contrast they integrated GCC 14.01 in F40 (still in beta until may), so no shy of early adoption of important changes.

It really would be usefull if they apply minor releases like any normal upstream patches in a timely fashion (e.g. google chrome, firefox, httpd ...).

3. Packaging and installation

For the normal user packaging and installation is not always obvious.
Postgres is split into different packages which can be installed separately. This is not documented well. There is no guide which packages should be installed to get a standard server like on windows or normal self compilation (including contrib, server, client, libs, libpq).

There is no information of how installing more then one major version on the same server (e.g. for pg_upgrade) interacts with each other (which libpq is in use?, which path is set?).

Removing an installation of an elder version (with more then one installed) leaves the path unset, so that the normal commands don't work as expected. (see my mail from february).

Further it is unclear if the installations bundled with the OS and the repositories provided by PGDG use the same packaging/tools and are fully interchangeable.

Is it possible to upgrade the pg version from OS distribution with the PGDG version on production without any hiccup?

4. Undocumented initialization behaviour

There are subtle differences in initializing the database.

I used to initialize the cluster with initdb and the appropriate flags, coming from windows installations.

Using the standard recommandation on Fedora:


PGSETUP_INITDB_OPTIONS=" --encoding=UTF8 --data-checksums --lc-messages=C --lc-collate=C"
export PGSETUP_INITDB_OPTIONS

sudo /usr/pgsql-16/bin/postgresql-16-setup initdb

I got a database without data checksum enabled!

It took me a long time to realize

-- no sudo, otherwise the options are not taken !!!

 to succesfull get

/usr/pgsql-16/bin/postgresql-16-setup initdb

That sudo prohibits the options to be promoted so that postgresql-16-setup did not see them.

I think postgresql-xx-setup (and other changes to the source tree behaviour like pg_hba,conf,ownership of files, which tools run under which user (server, clients applications), priviliges for writing to the file system with server side copy, wal archive copying, firewall settings, service enabling, selinux implications etc.) should be fully documented and the user should be given a real example of its usage, since this is nowhere present in the source tree and unknown for peoble coming from e.g. windows.

5. Improving usability
All these points are from my real world experience of usablity of Postgres. I have managed to learn or circumvent specific aspects, but I think usability for every not so skilled user should be in the main focus.

Please see my suggestions this way!

Hans Buschmann




Von: Devrim Gündüz <devrim@gunduz.org>
Gesendet: Freitag, 3. Mai 2024 01:44
An: Hans Buschmann; pgsql-hackers@postgresql.org
Betreff: Re: Proposal: Early providing of PGDG repositories for the major Linux distributions like Fedora or Debian
 
Hi,

On Wed, 2024-04-24 at 11:02 +0000, Hans Buschmann wrote:
> Today (24.4.2024) I upgraded my laptop to Fedora 40, but there where
> no repository available, so I ended with a mix of Fedora 40 and Fedora
> 39 installation.

This was caused by an unexpected and very long network outage that
blocked my access to the build instances. I released packages last
Thursday (2 days after F40 was released, which is 26 April 2024)

I sent emails to the RPM mailing list about the issue:

https://www.postgresql.org/message-id/aec36aec623741ae314692b318c890c646498ca6.camel%40gunduz.org
https://www.postgresql.org/message-id/1fe99b0def5d7539939421fa5b35db2c8f2a40ad.camel%40gunduz.org
https://www.postgresql.org/message-id/3a1b0f58673d35fae9979ed2b149972195c7b8bc.camel%40gunduz.org

> To mitigate this situation, I propose to provide these repositories
> much earlier in the beta phase of the distributions:

This is what I do for the Fedora releases. I'm sure you've noticed that
in the past.

Regards,

--
Devrim Gündüz
Open Source Solution Architect, PostgreSQL Major Contributor
Twitter: @DevrimGunduz , @DevrimGunduzTR