Re: Designing a better connection pool for psycopg3

Поиск
Список
Период
Сортировка
От Vladimir Ryabtsev
Тема Re: Designing a better connection pool for psycopg3
Дата
Msg-id CAMqTPqkx0rx2a79rV7UOhs1CxXZG+vG6F_qV2QWR4+9Bd+1Ocg@mail.gmail.com
обсуждение исходный текст
Ответ на Designing a better connection pool for psycopg3  (Daniele Varrazzo <daniele.varrazzo@gmail.com>)
Ответы Re: Designing a better connection pool for psycopg3  (Karsten Hilbert <Karsten.Hilbert@gmx.net>)
Re: Designing a better connection pool for psycopg3  (Karsten Hilbert <Karsten.Hilbert@gmx.net>)
Re: Designing a better connection pool for psycopg3  (Karsten Hilbert <Karsten.Hilbert@gmx.net>)
Список psycopg
> Connection configuration
In my opinion it is not only static vs dynamic,
I have a feeling that subclassing gives more freedom for improper
implementation (resulting in problems). Also it depends on internal
API (not necessarily public), which puts more obligations on the
maintainer to keep it compatible with old releases. In this case
I like the configuring approach better
(and it does not exclude subclassing as well).

> Exclusive connections?
The feature itself seems useful, but what are the benefits of getting it
from the pool (e.g. checking state, reconnection, configuration)
as opposed to creating one manually?
Is such a connection counted in regard to 'minconn', 'maxconn'?

> What to do in case of connection failure?
> A first step towards a sane behaviour could be to die early, on startup, if the connection is not working when the first pool population is done
Agree.
BTW, at which moment will the driver be trying to connect? At pool
initialization or on first "getconn()"?

> What if the database is missing in action during the program lifetime? Following connection attempts may be repeated, with an exponential backoff, until dying after a few minutes of fruitless attempts
Yes, I would prefer exponential backoff, with a timeout prior to "panic" callback.
Preferably, both configurable.

> Connections usage pattern
I don't see cases where an application should bother about it,
choose whatever is better for your implementation. And yes, stack
looks reasonable.

> 'minconn', Proposed default: 4
Many services and applications do not really need many connections
but would like to benefit from automatic reconnections. So the default
is questionable, as many people will leave it unchanged.

> "get_info()"
One thing that comes into my head every time I use connection pools
is how many are really needed for my application. Usually it is adjusted
experimentally or using some performance tests. Can you collect and
provide some stats to get more insights on that? I see you want to
expose some in "get_info()", which exactly? E.g. number of times
the max limit was hit, current number of connections acquired?

> 'max_lifetime_sec'
Why is it necessary at all? Does a connection get rotten with time?
As I understand, you are planning to check a connection state in
background. As long as it is OK, what is the need to replace it?

> num_workers, default: 3
Assuming every worker does the same job, is there really the amount
of work for three workers?

"get_maintenance_task()"
This API seems to be too advanced, in what cases you see a sane use case for it?

Thank you,
Vladimir


On Mon, 18 Jan 2021 at 04:04, Daniele Varrazzo <daniele.varrazzo@gmail.com> wrote:
Hello,

I've been gathering a few ideas about the connection pool I would like
to provide with psycopg3. I would be happy if you would like to take a
look and give some feedback.

https://www.psycopg.org/articles/2021/01/17/pool-design/

Thank you very much,

-- Daniele


В списке psycopg по дате отправления:

Предыдущее
От: Aryeh Leib Taurog
Дата:
Сообщение: pgcopy 1.5.0 release announcement
Следующее
От: Karsten Hilbert
Дата:
Сообщение: Re: Designing a better connection pool for psycopg3