Обсуждение: [PATCHES] pglister & pgarchives: add deployment tests for gitlab-ci

Since I have an ansible script [1] to deploy pglister and pgarchives, and to avoid pushing breaking commits into
master,I've worked on adding a full deployment test to gitlab-ci.
 

In this series of patches for pglister:
0001. adds a deployment stage for gitlab-ci
0002. updates the tools used in the syntax check stage
0003. Fixes new errors found by the updated version of pep8/pycodestyle

Pgarchives patches in the following email.

Example test runs:
- pglister: https://gitlab.com/cmatte/pglister/-/jobs/4890819780
- pgarchives: https://gitlab.com/cmatte/pgarchives/-/jobs/4890879735

[1]: https://gitlab.com/cmatte/ansible-pglister.git
-- 
Célestin Matte

The patches for pgarchives:
0001. convert two scripts in loader/tools to python3 so that they pass the syntax check part
0002. add the gitlab-ci file, with both a syntax and a deployment stage, to follow the model of pglister

For this to work, the pgarchives repository would have to be moved to gitlab as it's not currently the case (I'd update
myansible-pglister script to the correct location).
 
-- 
Célestin Matte

On Thu, Aug 17, 2023 at 11:58 AM Célestin Matte
<celestin.matte@cmatte.me> wrote:
>
> Since I have an ansible script [1] to deploy pglister and pgarchives, and to avoid pushing breaking commits into
master,I've worked on adding a full deployment test to gitlab-ci. 
>
> In this series of patches for pglister:
> 0001. adds a deployment stage for gitlab-ci
> 0002. updates the tools used in the syntax check stage
> 0003. Fixes new errors found by the updated version of pep8/pycodestyle
>
> Pgarchives patches in the following email.
>
> Example test runs:
> - pglister: https://gitlab.com/cmatte/pglister/-/jobs/4890819780
> - pgarchives: https://gitlab.com/cmatte/pgarchives/-/jobs/4890879735
>
> [1]: https://gitlab.com/cmatte/ansible-pglister.git


0001 - I'm unsure if we want to hard-code something that relies on an
external repo like that into the pipelines. But that type of check in
general can make sense. But what type of errors would you expect it to
detect?

0002 - As I mentioned on the other thread (I think), our primary
platform is still buster. We shouldn't change the toolchain to
bookworm until we are ready to do it everywhere, and we are ont yet.
The part changing it to pycodestyle I think is valid on buster as well
though?

0003 - This one has already been merged right? It was already
submitted separately?

pgarchives patches:

0001 - It seems to convert the contents of the files to python3, but
the shebang still refers python2? I assume that's just an oversight?

0002 - Same basic comment applies, but also, pgarchives is not
currently hosted at gitlab, so the gitlab-ci scripts won't do much?


//Magnus

> 0001 - I'm unsure if we want to hard-code something that relies on an
> external repo like that into the pipelines. But that type of check in
> general can make sense. But what type of errors would you expect it to
> detect?

My script performs a full install, i.e. installing httpd, exim, postgresql, configuring them appropriately and running
thedjango install.
 
The main point of running this is to check that the django install still works on a fresh install. For instance, this
wouldhave automatically detected the issue introduced in 1527758 and fixed in 749be96.
 
Granted, this can be a bit overkill for the end purpose. We could either reduce it to testing the django install only,
oradd more tests (e.g. testing test process of subscription, getting messages published etc.)
 
What do you think?

> 0002 - As I mentioned on the other thread (I think), our primary
> platform is still buster. We shouldn't change the toolchain to
> bookworm until we are ready to do it everywhere, and we are ont yet.
> The part changing it to pycodestyle I think is valid on buster as well
> though?

OK, patch for changing to pycodestyle only attached (it works: https://gitlab.com/cmatte/pglister/-/jobs/5136488484)

> 0003 - This one has already been merged right? It was already
> submitted separately?

Yes, apologies, I ended up sending it in two separate threads

> pgarchives patches:
> 
> 0001 - It seems to convert the contents of the files to python3, but
> the shebang still refers python2? I assume that's just an oversight?

Indeed, attached

> 0002 - Same basic comment applies, but also, pgarchives is not
> currently hosted at gitlab, so the gitlab-ci scripts won't do much?

Yes, as stated in the email, the pgarchives repository would need to be moved to gitlab for this to work (why isn't it
already,similarly to pglister?)
 

-- 
Célestin Matte