Re: reindex/vacuum locking/performance?

Stepping out on a limb... (I'm not a disk kernel guy)

I have long thought that as part of a cache descriptor, there should be a
process-definable replacement-strategy (RS). Each cache entry would be
associated to each process's replacement-strategy variable and the
page-replacement algorithm would then take into consideration the desired
policy. Imagine for simplicity sake, that each strategy gets its own cache
table. When it comes time to replace a page, the system scans the cache
tables, picks the most likely page for replacement from each table, then
selects the most likely page between all policies. This allows the 99% of
apps that can make excellent use of use LRU to use LRU among themselves
(best for everyone), and the MRU (better for databases) (best for everyone
too) to only sacrifice the best pages between MRU apps. Though, once you
have an MRU process, the final decision between taking the page should be
use MRU, and not LRU. Of course there are a number of questions: does each
RS get its own table, to be managed independently, or can we combine them
all into one table? What are the performance implications of the multiple
table management?

One day, I'd like to see function pointers and kernel modules used as ways
for apps to manage replacement policy. fantasyland# insmod MRU.o
fantasyland# vi postgresql.conf { replacement_policy=MRU }
{meanwhile in some postgre .c file:}
set_cache_policy(get_cfg_replacement_policy());
fantasyland# service postmaster restart

Anyone want to throw this at the kernel developers?

> -----Original Message-----
> From: pgsql-performance-owner@postgresql.org
> [mailto:pgsql-performance-owner@postgresql.org]On Behalf Of Tom Lane
> Sent: Monday, October 06, 2003 2:26 PM
> To: Neil Conway
> Cc: Andrew Sullivan; PostgreSQL Performance
> Subject: Re: [PERFORM] reindex/vacuum locking/performance?
>
>
> Neil Conway <neilc@samurai.com> writes:
> > On Sun, 2003-10-05 at 19:50, Neil Conway wrote:
> > I was hoping you'd reply to this, Tom -- you were referring to O_DIRECT,
> > right?
>
> Not necessarily --- as you point out, it's not clear that O_DIRECT would
> help us.  What would be way cool is something similar to what James
> Rogers was talking about: a way to tell the kernel not to promote this
> page all the way to the top of its LRU list.  I'm not sure that *any*
> Unixen have such an API, let alone one that's common across more than
> one platform :-(
>
>             regards, tom lane
>
> ---------------------------(end of broadcast)---------------------------
> TIP 3: if posting/reading through Usenet, please send an appropriate
>       subscribe-nomail command to majordomo@postgresql.org so that your
>       message can get through to the mailing list cleanly
>