Re: Overriding the optimizer

Tom,

>>I see this over and over.  Tweak the parameters to "force" a certain
>>plan, because there's no formal way for a developer to say, "I know
>>the best plan."
>
> I think you've misunderstood those conversations entirely.  The point
> is not to force the planner into a certain plan, it is to explore what's
> going on with a view to understanding why the planner isn't making a
> good choice, and thence hopefully improve the planner in future.

No, I understood the conversations very clearly.  But no matter how clever the optimizer, it simply can't compete with
adeveloper who has knowledge that Postgres *can't* have.  The example of a user-written function is obvious. 

>>There isn't a database in the world that is as smart as a developer,
>
> People who are convinced they are smarter than the machine are often
> wrong ;-).

Often, but not always -- as I noted in my original posting.  And when the developer is smarter than Postgres, and
Postgresmakes the wrong choice, what is the developer supposed to do?  This isn't academic -- the wrong plans Postgres
makescan be *catastrophic*, e.g. turning a 3-second query into a three-hour query. 

How about this: Instead of arguing in the abstract, tell me in concrete terms how you would address the very specific
exampleI gave, where myfunc() is a user-written function.  To make it a little more challenging, try this: myfunc() can
behavevery differently depending on the parameters, and sometimes (but not always), the application knows how it will
behaveand could suggest a good execution plan. 

(And before anyone suggests that I rewrite myfunc(), I should explain that it's in the class of NP-complete problems.
Thefunction is inherently hard and can't be made faster or more predictable.) 

The example I raised in a previous thread, of irregular usage, is the same: I have a particular query that I *always*
wantto be fast even if it's only used rarely, but the system swaps its tables out of the file-system cache, based on
"lowusage", even though the "high usage" queries are low priority.  How can Postgres know such things when there's no
wayfor me to tell it? 

The answers from the Postgres community were essentially, "Postgres is smarter than you, let it do its job."
Unfortunately,this response completely ignores the reality: Postgres is NOT doing its job, and can't, because it
doesn'thave enough information. 

Craig