On Wed, 2023-06-21 at 18:53 -0700, Wells Oliver wrote:
> Hi all. Trying to asses our WAL checkpoint during heavy ETL loads.
> Checkpoint timer is at 5min and it seems to be constantly checkpointing, maybe that's OK.
> We see an increased % of buffer percentages in the logs during the checkpoint. Here's a run:
>
> [...]
>
> I don't want to read too much into it
But that's how you learn...
> but I am trying to understand it better to consider any potential changes in max WAL sizes,
> or increasing our checkpoint timeouts if they are causing the ETL stuff to be slower than
> it should. Our wal_level is currently set to logical but we plan to change this to replica
> soon as it feels like overkill.
You should increase "max_wal_size" until most of your checkpoints are triggered by
timeout, once every 5 minutes. That will reduce the amount of WAL, because you will
have fewer full-page images, and you will avoid redundant buffer writes.
The only negative side effect is that recovery can take longer if you crash while
loading.
Yours,
Laurenz Albe