Re: Word-smithing doc changes

Excerpts from Greg Stark's message of sáb jun 25 21:01:36 -0400 2011:
>>> I think this commit was ill-advised:
>>> http://git.postgresql.org/gitweb?p=postgresql.git;a=commitdiff;h=a03feb9354bda5084f19cc952bc52ba7be89f372
>>> Seems way to implementation-specific and detailed for a user to make
>>> heads or tails of. Except in the sections talking about locking
>>> internals we don't talk about "shared locks on virtual transactions
>>> identifiers" we just talk about waiting for a transaction to complete.

Looks like I missed this when it passed by before, and looks like Greg 
Stark may have missed the message on pgsql-docs that kicked this all 
off:  
http://archives.postgresql.org/message-id/4DDB64CB.7070109@2ndQuadrant.com

I will happily accept that the description there may have suffered from 
me not using all of the terms optimally, and that the resulting commit 
could be improved.  Some more feedback to get the description correct 
and useful would be much appreciated.

What I cannot agree with is that idea that the implementation details I 
suggested documenting should not be.  There are extremely user-hostile 
things that can happen here, and that are unique to this command.  
Saying "this is too complicated for users to make heads or tails of" may 
very well be true in many cases, but I think it's not giving PostgreSQL 
users very much credit.  And when problems with this happen, and I 
wouldn't have spent any time on this if they didn't, right now the only 
way to make heads or tails of it is to read the source code.

If the code was simple, quick, and had no failure modes, it would be 
fine to not describe it.  This is complicated, the run time cannot be 
bounded, and it can ripple to nasty lock queue issues--at some 
impossible to predict future time, long after you start the creation.  I 
don't have a good idea how to unload the potential foot gun.  The best I 
could think of after being shot with it was describing how it fires.

>>> And looping over the transactions one by one is purely an
>>> implementation detail and uninteresting to users.

That particular suggestion came from me having a painful session I 
didn't want anyone else to ever go through again.  By the end of that, 
this implementation detail felt like the most important missing piece of 
PostgreSQL documentation in the world to me--I'm too busy to send in doc 
patches describing things that I haven't been shot by.

To provide some more context, the server I ran into this on always has 
at least 2 reports that take 10 to 16 hours to run active.  I happily 
kicked off a concurrent index build on a heavily bloated 1GB table whose 
indexes are typically >5GB, which I expect to take a few minutes given 
the small size involved.  Six hours later, when I come back and discover 
it's still not done, I find a single lock waiting for a transaction to 
finish.    Since I'm used to multiple locks forming into a tree 
structure, and I only see one, I expect I'm OK once that's done.  Fine; 
I estimate how much time that report has left and leave for a bit.

Four hours later, I come back.  That original transaction lock is gone.  
Now it's created a new one I didn't expect, moving onto the second 
oldest report active.  I actually have six more hours to go still.  This 
locking pattern is unique to this command, and if I'd had the slightest 
idea that it worked this way I'd have approached the rebuild differently.