Обсуждение: Regarding GIN Fast Update Technique

Поиск
Список
Период
Сортировка

Regarding GIN Fast Update Technique

От
Amit Langote
Дата:
Hello,

At what point do the entries in the pending list are moved to the main
GIN data structure?
From documentation, I read that overflowing work_mem and vacuum are
two such causes; what about when the concerned backend is to exit and
autovacuum has not yet kicked in?


--
Amit Langote



Re: Regarding GIN Fast Update Technique

От
Robert Haas
Дата:
On Wed, Jun 5, 2013 at 10:06 PM, Amit Langote <amitlangote09@gmail.com> wrote:
> At what point do the entries in the pending list are moved to the main
> GIN data structure?
> From documentation, I read that overflowing work_mem and vacuum are
> two such causes; what about when the concerned backend is to exit and
> autovacuum has not yet kicked in?

I don't think there's any special handling for that case, nor do I
think any is needed.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company



Re: Regarding GIN Fast Update Technique

От
Amit Langote
Дата:
On Fri, Jun 7, 2013 at 11:15 PM, Robert Haas <robertmhaas@gmail.com> wrote:
> On Wed, Jun 5, 2013 at 10:06 PM, Amit Langote <amitlangote09@gmail.com> wrote:
>> At what point do the entries in the pending list are moved to the main
>> GIN data structure?
>> From documentation, I read that overflowing work_mem and vacuum are
>> two such causes; what about when the concerned backend is to exit and
>> autovacuum has not yet kicked in?
>
> I don't think there's any special handling for that case, nor do I
> think any is needed.
>

Okay, aside from that case, what else would move those to the main
structure? They (the entries in the unsorted pending list) are in the
local memory (work_mem?) of the backend, right?


--
Amit Langote



Re: Regarding GIN Fast Update Technique

От
Andres Freund
Дата:
On 2013-06-07 23:28:56 +0900, Amit Langote wrote:
> On Fri, Jun 7, 2013 at 11:15 PM, Robert Haas <robertmhaas@gmail.com> wrote:
> > On Wed, Jun 5, 2013 at 10:06 PM, Amit Langote <amitlangote09@gmail.com> wrote:
> >> At what point do the entries in the pending list are moved to the main
> >> GIN data structure?
> >> From documentation, I read that overflowing work_mem and vacuum are
> >> two such causes; what about when the concerned backend is to exit and
> >> autovacuum has not yet kicked in?
> >
> > I don't think there's any special handling for that case, nor do I
> > think any is needed.
> >
> 
> Okay, aside from that case, what else would move those to the main
> structure? They (the entries in the unsorted pending list) are in the
> local memory (work_mem?) of the backend, right?

No, it's in the normal heap, pointed to by the metapage. Storing it in
local memory would mean we would have to flush it out before commit.

That part of gin is actually quite readable code, so I suggest looking
there. Start with ginfast.c:ginHeapTupleFastInsert().

Greetings,

Andres Freund

-- Andres Freund                       http://www.2ndQuadrant.com/PostgreSQL Development, 24x7 Support, Training &
Services



Re: Regarding GIN Fast Update Technique

От
Tom Lane
Дата:
Amit Langote <amitlangote09@gmail.com> writes:
> Okay, aside from that case, what else would move those to the main
> structure? They (the entries in the unsorted pending list) are in the
> local memory (work_mem?) of the backend, right?

No.  If they were, it wouldn't be crash-safe.
        regards, tom lane



Re: Regarding GIN Fast Update Technique

От
Amit Langote
Дата:
On Fri, Jun 7, 2013 at 11:36 PM, Andres Freund <andres@2ndquadrant.com> wrote:
> On 2013-06-07 23:28:56 +0900, Amit Langote wrote:
>> On Fri, Jun 7, 2013 at 11:15 PM, Robert Haas <robertmhaas@gmail.com> wrote:
>> > On Wed, Jun 5, 2013 at 10:06 PM, Amit Langote <amitlangote09@gmail.com> wrote:
>> >> At what point do the entries in the pending list are moved to the main
>> >> GIN data structure?
>> >> From documentation, I read that overflowing work_mem and vacuum are
>> >> two such causes; what about when the concerned backend is to exit and
>> >> autovacuum has not yet kicked in?
>> >
>> > I don't think there's any special handling for that case, nor do I
>> > think any is needed.
>> >
>>
>> Okay, aside from that case, what else would move those to the main
>> structure? They (the entries in the unsorted pending list) are in the
>> local memory (work_mem?) of the backend, right?
>
> No, it's in the normal heap, pointed to by the metapage. Storing it in
> local memory would mean we would have to flush it out before commit.
>
> That part of gin is actually quite readable code, so I suggest looking
> there. Start with ginfast.c:ginHeapTupleFastInsert().
>

Thanks.

--
Amit Langote



Re: Regarding GIN Fast Update Technique

От
Jesper Krogh
Дата:
On 07/06/13 16:39, Tom Lane wrote:
> Amit Langote <amitlangote09@gmail.com> writes:
>> Okay, aside from that case, what else would move those to the main
>> structure? They (the entries in the unsorted pending list) are in the
>> local memory (work_mem?) of the backend, right?
> No.  If they were, it wouldn't be crash-safe.
Thats how it is, but if we could push in wishes, then I would
wish that is woulndn't be crash-safe, and be flushed by the backends
commit. The way it currently operates is that a "random backend"
pays the penalty of other backends pushes to the pending-list and "all 
queries"
pays the penalty of searching the pendinglist in queries.

If the pending list were backend only it would not have to be searched
by "all queries" since commit needs to flush it and random backends
wouldn't be penalized. Allthough we'd still have the benefit of batching up
gin-inserts over mulitiple changes to the index done within the same
transaction.

Jesper

-- 
Jesper