Re: parallelizing the archiver

Поиск
Список
Период
Сортировка
От Stephen Frost
Тема Re: parallelizing the archiver
Дата
Msg-id 20210914201458.GH17906@tamriel.snowman.net
обсуждение исходный текст
Ответ на Re: parallelizing the archiver  (Julien Rouhaud <rjuju123@gmail.com>)
Ответы Re: parallelizing the archiver  (Julien Rouhaud <rjuju123@gmail.com>)
Список pgsql-hackers
Greetings,

* Julien Rouhaud (rjuju123@gmail.com) wrote:
> On Fri, Sep 10, 2021 at 2:03 PM Andrey Borodin <x4mmm@yandex-team.ru> wrote:
> > > 10 сент. 2021 г., в 10:52, Julien Rouhaud <rjuju123@gmail.com> написал(а):
> > > Yes, but it also means that it's up to every single archiving tool to
> > > implement a somewhat hackish parallel version of an archive_command,
> > > hoping that core won't break it.

We've got too many archiving tools as it is, if you want my 2c on that.

> > I'm not proposing to remove existing archive_command. Just deprecate it one-WAL-per-call form.
>
> Which is a big API beak.

We definitely need to stop being afraid of this.  We completely changed
around how restores work and made pretty much all of the backup/restore
tools have to make serious changes when we released v12.

I definitely don't think that we should be making assumptions that
changing archive command to start running things in parallel isn't
*also* an API break too, in any case.  It is also a change and there's
definitely a good chance that it'd break some of the archivers out
there.  If we're going to make a change here, let's make a sensible one.

> > It's a very simplistic approach. If some GUC is set - archiver will just feed ready files to stdin of archive
command.What fundamental design changes we need? 

Haven't really thought about this proposal but it does sound
interesting.

Thanks,

Stephen

Вложения

В списке pgsql-hackers по дате отправления:

Предыдущее
От: Tom Lane
Дата:
Сообщение: Re: Getting ERROR "subplan "SubPlan 1" was not initialized" in EXISTS subplan when using for list partition.
Следующее
От: Jelte Fennema
Дата:
Сообщение: Re: [EXTERNAL] Re: Don't clean up LLVM state when exiting in a bad way