Re: Optimize planner memory consumption for huge arrays

Поиск
Список
Период
Сортировка
От Tomas Vondra
Тема Re: Optimize planner memory consumption for huge arrays
Дата
Msg-id b34b5fa6-2f35-4b49-823d-94e61aa0368d@enterprisedb.com
обсуждение исходный текст
Ответ на Re: Optimize planner memory consumption for huge arrays  (Tom Lane <tgl@sss.pgh.pa.us>)
Ответы Re: Optimize planner memory consumption for huge arrays  (Tom Lane <tgl@sss.pgh.pa.us>)
Список pgsql-hackers

On 2/25/24 17:29, Tom Lane wrote:
> Tomas Vondra <tomas.vondra@enterprisedb.com> writes:
>> On 2/25/24 00:07, Tom Lane wrote:
>>> ...  I'm not sure if it'd be worth extending
>>> the mcxt.c API to provide something like "MemoryContextResetIfBig",
>>> with some internal rule that would be cheap to apply like "reset
>>> if we have any non-keeper blocks".
> 
>> I think MemoryContextResetIfBig is an interesting idea - I think a good
>> way to define "big" would be "has multiple blocks", because that's the
>> only case where we can actually reclaim some memory.
> 
> Yeah.  Also: once we had such an idea, it'd be very tempting to apply
> it to other frequently-reset contexts like the executor's per-tuple
> evaluation contexts.  I'm not quite prepared to argue that
> MemoryContextReset should just act that way all the time ... but
> it's sure interesting to think about.
> 

Do the context resets consume enough time to make this measurable? I may
be wrong, but I'd guess it's not measurable. In which case, what would
be the benefit?

> Another question is whether this wouldn't hurt debugging, in that
> dangling-pointer bugs would become harder to catch.  We'd certainly
> want to turn off the optimization in USE_VALGRIND builds, and maybe
> we just shouldn't do it at all if USE_ASSERT_CHECKING.
> 
>             regards, tom lane

+1 to disable this optimization in assert-enabled builds. I guess we'd
invent a new constant to disable it, and tie it to USE_ASSERT_CHECKING
(similar to CLOBBER_FREED_MEMORY, for example).

Thinking about CLOBBER_FREED_MEMORY, could it be useful to still clobber
the memory, even if we don't actually reset the context?


regards

-- 
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company



В списке pgsql-hackers по дате отправления:

Предыдущее
От: Japin Li
Дата:
Сообщение: Re: Improve readability by using designated initializers when possible
Следующее
От: "Andrey M. Borodin"
Дата:
Сообщение: Re: Injection points: some tools to wait and wake