Обсуждение: documentation on HOT
Hi, While working on the 2020-02-10 release announcement, in order to better describe one of the fixes, I tried to find a reference in the docs that described what HOT is, how it works, etc. in a user-friendly manner. This lead me to the acronyms page[1], which lead me to a document in the repo[2]. The closest thing I found was a blog post that Laurenz Albe wrote on the topic[3]. Given the importance of HOT, it seems like this would be a good topic to document. I would suggest something higher-level for general users in the "Indexes"[4] section, and something lower-level in internals[5] (which could perhaps be derived from [2]). Thoughts on this? Thanks, Jonathan [1] https://www.postgresql.org/docs/current/acronyms.html [2] https://git.postgresql.org/gitweb/?p=postgresql.git;a=blob;f=src/backend/access/heap/README.HOT;hb=HEAD [3] https://www.cybertec-postgresql.com/en/hot-updates-in-postgresql-for-better-performance/ [4] https://www.postgresql.org/docs/current/indexes.html [5] https://www.postgresql.org/docs/current/internals.html
Вложения
On Sun, Feb 6, 2022 at 6:08 PM Jonathan S. Katz <jkatz@postgresql.org> wrote:
Given the importance of HOT, it seems like this would be a good topic to
document. I would suggest something higher-level for general users in
the "Indexes"[4] section, and something lower-level in internals[5]
(which could perhaps be derived from [2]).
Thoughts on this?
I'm doubting there is any disagreement that this is needed. Three of us said as much recently [1] while discussing a user question regarding our documentation for expression indexes.
I presently haven't felt moved to fill the need myself as so I was fine with at least providing a high-level summary in the glossary which we also the general user to understand the idea without having to dive into the README. The details for internals can continue to be handled there in the interest of at least getting something committed for the majority of users.
David J.
On 2/6/22 8:56 PM, David G. Johnston wrote: > On Sun, Feb 6, 2022 at 6:08 PM Jonathan S. Katz <jkatz@postgresql.org > <mailto:jkatz@postgresql.org>> wrote: > > > Given the importance of HOT, it seems like this would be a good > topic to > document. I would suggest something higher-level for general users in > the "Indexes"[4] section, and something lower-level in internals[5] > (which could perhaps be derived from [2]). > > Thoughts on this? > > > I'm doubting there is any disagreement that this is needed. Three of us > said as much recently [1] while discussing a user question regarding our > documentation for expression indexes. I did try to search for previous discussion on this (not well enough apparently) and I was unable to find the thread. That is also part of the challenge with the term HOT. > I presently haven't felt moved to fill the need myself as so I was fine > with at least providing a high-level summary in the glossary which we > also the general user to understand the idea without having to dive into > the README. I don't agree that such an explanation belongs in the glossary. That feels like it would be too brief, and I think that does a disservice to the topic and our users. I agree with Bruce's point that we should have a new section (or subsection). As I mentioned in my previous post, given HOT involves indexing, I would suggest putting it there. I think that something that follows the general outline of Laurenz's post would satisfy the user requirements. It explains at a high level what HOT is, it's advantages, and how it works. > The details for internals can continue to be handled there > in the interest of at least getting something committed for the majority > of users. I do think there should be some reference of it to the docs. Even if we have a page in the "Internals" section that says "For more information on how HOT works, please see <link>". That said, is there any reasoning why the HOT README (or something similar to it) is not in the "Internals" section of the documentation, similar to other indexing topics? Thanks, Jonathan
Вложения
On Sun, Feb 6, 2022 at 09:29:56PM -0500, Jonathan Katz wrote: > I agree with Bruce's point that we should have a new section (or > subsection). As I mentioned in my previous post, given HOT involves > indexing, I would suggest putting it there. > > I think that something that follows the general outline of Laurenz's post > would satisfy the user requirements. It explains at a high level what HOT > is, it's advantages, and how it works. Attached is a patch that adds a new HOT section to the storage chapter. You can see the results here: https://momjian.us/tmp/pgsql/storage-hot.html -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com Indecision is a decision. Inaction is an action. Mark Batterson
Вложения
On Thu, Jul 21, 2022 at 10:02:18PM -0400, Bruce Momjian wrote: > On Sun, Feb 6, 2022 at 09:29:56PM -0500, Jonathan Katz wrote: > > I agree with Bruce's point that we should have a new section (or > > subsection). As I mentioned in my previous post, given HOT involves > > indexing, I would suggest putting it there. > > > > I think that something that follows the general outline of Laurenz's post > > would satisfy the user requirements. It explains at a high level what HOT > > is, it's advantages, and how it works. > > Attached is a patch that adds a new HOT section to the storage chapter. I wasn't happy with the last paragraph so I added some more details. URL contents updated too: https://momjian.us/tmp/pgsql/storage-hot.html -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com Indecision is a decision. Inaction is an action. Mark Batterson
Вложения
On 7/22/22 8:51 AM, Bruce Momjian wrote: > On Thu, Jul 21, 2022 at 10:02:18PM -0400, Bruce Momjian wrote: >> On Sun, Feb 6, 2022 at 09:29:56PM -0500, Jonathan Katz wrote: >>> I agree with Bruce's point that we should have a new section (or >>> subsection). As I mentioned in my previous post, given HOT involves >>> indexing, I would suggest putting it there. >>> >>> I think that something that follows the general outline of Laurenz's post >>> would satisfy the user requirements. It explains at a high level what HOT >>> is, it's advantages, and how it works. >> >> Attached is a patch that adds a new HOT section to the storage chapter. > > I wasn't happy with the last paragraph so I added some more details. > URL contents updated too: > > https://momjian.us/tmp/pgsql/storage-hot.html Thanks! This is great. Probably the most concise and clear explanation I have seen for HOT, which is exactly what we need for the docs :) A few suggestions: s/Fortunately, there is/To help reduce overhead, PostgreSQL has/ s/In summary, h/H/ > You can increase the change of HOT updates by using non-default table fillfactor settings. I think we should expand on this and explain how adjusting "fillfactor" will affect this. I think that may change the final sentence too. Thanks, Jonathan
Вложения
On Fri, Jul 22, 2022 at 8:09 AM Jonathan S. Katz <jkatz@postgresql.org> wrote:
On 7/22/22 8:51 AM, Bruce Momjian wrote:
> On Thu, Jul 21, 2022 at 10:02:18PM -0400, Bruce Momjian wrote:
>> On Sun, Feb 6, 2022 at 09:29:56PM -0500, Jonathan Katz wrote:
>>> I agree with Bruce's point that we should have a new section (or
>>> subsection). As I mentioned in my previous post, given HOT involves
>>> indexing, I would suggest putting it there.
>>>
>>> I think that something that follows the general outline of Laurenz's post
>>> would satisfy the user requirements. It explains at a high level what HOT
>>> is, it's advantages, and how it works.
>>
>> Attached is a patch that adds a new HOT section to the storage chapter.
>
> I wasn't happy with the last paragraph so I added some more details.
> URL contents updated too:
>
> https://momjian.us/tmp/pgsql/storage-hot.html
Thanks! This is great. Probably the most concise and clear explanation I
have seen for HOT, which is exactly what we need for the docs :)
A few suggestions:
s/Fortunately, there is/To help reduce overhead, PostgreSQL has/
s/In summary, h/H/
> You can increase the change of HOT updates by using non-default table
fillfactor settings.
I think we should expand on this and explain how adjusting "fillfactor"
will affect this. I think that may change the final sentence too.
I think we need to expose the information regarding columns used in predicates here.
"(Here, "indexed column" means any column referenced
at all in an index definition, including for example columns that are
tested in a partial-index predicate but are not stored in the index.)"
at all in an index definition, including for example columns that are
tested in a partial-index predicate but are not stored in the index.)"
I get it is an implementation detail but explaining the name seems like a good thing to do as well:
"Without HOT, every version of a row in an update chain has its own index
entries, even if all indexed columns are the same. With HOT, a new tuple
placed on the same page and with all indexed columns the same as its
parent row version does not get new index entries. This means there is
only one index entry for the entire update chain on the heap page.
An index-entry-less tuple is marked with the HEAP_ONLY_TUPLE flag."
entries, even if all indexed columns are the same. With HOT, a new tuple
placed on the same page and with all indexed columns the same as its
parent row version does not get new index entries. This means there is
only one index entry for the entire update chain on the heap page.
An index-entry-less tuple is marked with the HEAP_ONLY_TUPLE flag."
Where the last sentence becomes: "Those index-entry-less tuples (yeah, still dislike triple-hypenation...) are thus named "Heap-Only Tuples".
(I've actually incorporated this as I think it should be down below, as a lead-in to the listing of conditions for when the optimization can be used.)
Then maybe "can be removed during select" should be reworded as:
"No longer visible heap-only tuples can be removed during normal
operation, including <command>SELECT</command>s, instead of requiring
periodic vacuum operations."
operation, including <command>SELECT</command>s, instead of requiring
periodic vacuum operations."
The original heap entry the index points to cannot be removed. "Old versions of heap-only tuples" vs. "No longer visible heap-only tuples" is probably a style choice. There are basically three different "versions" in context here though so avoiding "old versions" has some appeal to me.
I'm not a fan of:
"Fortunately, there is an automatic system..."
I'd like to give credit to the fact we engineered a solution to the downsides, so change the lead-in paragraph to the conditions listing to be:
"To mitigate these downsides PostgreSQL implements an optimization whereby sometimes only the heap tuple is created, not the index entry, when performing an update. In a case of giving things obvious and meaningful names, this is the Heap-Only Tuple (HOT) Optimization. This optimization is possible when:"
David J.
On Fri, Jul 22, 2022 at 8:09 AM Jonathan S. Katz <jkatz@postgresql.org> wrote: > Thanks! This is great. Probably the most concise and clear explanation I > have seen for HOT, which is exactly what we need for the docs :) I'm delighted that this is finally happening! > I think we should expand on this and explain how adjusting "fillfactor" > will affect this. I think that may change the final sentence too. Definitely -- this is by far the most important reason to tune heap fill factor, which can make a big difference. There should be a link from the fill factor docs to the new HOT section, at a minimum. Another thing: would be good to point out how to monitor the number of HOT updates vs non-HOT updates using views like pg_stat_user_tables. One minor correction: Opportunistic pruning isn't limited to heap-only tuples -- "HOT pruning" is actually a bit of a misnomer, that somehow caught on in the Postgres community. Opportunistic pruning can and will happen even with non-HOT updates -- the only difference on the heap page level is that 4 byte LP_DEAD line pointers will accumulate over time, which only VACUUM is able to clean up (we need LP_DEAD items to stick around until then as tombstones, so that index scans don't ever get confused). But pruning will still be able to free most of the space on the heap page, in almost the same way. Of course the indexes on the table are another matter -- each and every index will accumulate versions when there are many non-HOT updates, which can be a huge problem. My point is that *from the point of view of an affected heap page*, pruning (whether opportunistic or performed by VACUUM) can go ahead without HOT tuples/updates, and isn't really that different in terms of the amount of space freed each time. Bottom-up index deletion is effective is no small part because pruning was always pretty effective even with non-HOT updates, except within indexes -- it pretty much compensates for that remaining problem. The heuristics that it uses in the nbtree code work quite well because of the natural tendency of those 4 byte LP_DEAD item pointers to concentrate on individual heap pages. We can fit so many more 4 byte LP_DEAD items than full sized live heap tuples that it becomes pretty obvious (from the point of view of the nbtree code that starts out by looking at an index leaf page) which pointed-to heap blocks are likely to enable it to free space via index tuple deletes -- it often visits the heap pages with the most TIDs. Before bottom-up index deletion was added, I would notice that pgbench variant workloads with non-HOT updates would result in a pgbench_accounts table that is maybe 1%-2% larger after a few hours, while the indexes on the same table would grow by 100% - 200%. Opportunistic pruning would work pretty well for the heap, but wouldn't do anything at all to control bloat in indexes. So opportunistic pruning was always very effective with non-HOT updates in one important way, but totally ineffective in another way. There is a section about bottom-up index deletion in the docs (in the B-Tree internals chapter) that already references HOT. It should link to this new chapter now, I think. -- Peter Geoghegan
On Fri, Jul 22, 2022 at 11:09:49AM -0400, Jonathan Katz wrote: > On 7/22/22 8:51 AM, Bruce Momjian wrote: > > URL contents updated too: > > > > https://momjian.us/tmp/pgsql/storage-hot.html > > Thanks! This is great. Probably the most concise and clear explanation I > have seen for HOT, which is exactly what we need for the docs :) > > A few suggestions: > > s/Fortunately, there is/To help reduce overhead, PostgreSQL has/ So, I can't do that without making the later part of the sentence redundant. I adjusted it in the attached patch. > > s/In summary, h/H/ Uh, I don't want to remove that since I am restating what I said above --- do you have another approach? > > You can increase the change of HOT updates by using non-default table > fillfactor settings. > > I think we should expand on this and explain how adjusting "fillfactor" will > affect this. I think that may change the final sentence too. Okay, done in the attached patch. -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com Indecision is a decision. Inaction is an action. Mark Batterson
Вложения
On Fri, Jul 22, 2022 at 09:25:43AM -0700, David G. Johnston wrote: > On Fri, Jul 22, 2022 at 8:09 AM Jonathan S. Katz <jkatz@postgresql.org> wrote: > I think we need to expose the information regarding columns used in predicates > here. > > "(Here, "indexed column" means any column referenced > at all in an index definition, including for example columns that are > tested in a partial-index predicate but are not stored in the index.)" Okay, I clarified this in the attached patch. > I get it is an implementation detail but explaining the name seems like a good > thing to do as well: > > "Without HOT, every version of a row in an update chain has its own index > entries, even if all indexed columns are the same. With HOT, a new tuple > placed on the same page and with all indexed columns the same as its > parent row version does not get new index entries. This means there is > only one index entry for the entire update chain on the heap page. > An index-entry-less tuple is marked with the HEAP_ONLY_TUPLE flag." I don't see how the chain is useful for people trying to understand how to benefit from this feature. > Where the last sentence becomes: "Those index-entry-less tuples (yeah, still > dislike triple-hypenation...) are thus named "Heap-Only Tuples". > > (I've actually incorporated this as I think it should be down below, as a > lead-in to the listing of conditions for when the optimization can be used.) > > Then maybe "can be removed during select" should be reworded as: > > "No longer visible heap-only tuples can be removed during normal > operation, including <command>SELECT</command>s, instead of requiring > periodic vacuum operations." I added a no-longer-visible qualifier to the patch. > The original heap entry the index points to cannot be removed. "Old versions of > heap-only tuples" vs. "No longer visible heap-only tuples" is probably a style > choice. There are basically three different "versions" in context here though > so avoiding "old versions" has some appeal to me. > > I'm not a fan of: > > "Fortunately, there is an automatic system..." > > I'd like to give credit to the fact we engineered a solution to the downsides, > so change the lead-in paragraph to the conditions listing to be: Yeah, good point. We didn't stumble upon this feature. I have adjusted that wording. > "To mitigate these downsides PostgreSQL implements an optimization whereby > sometimes only the heap tuple is created, not the index entry, when performing > an update. In a case of giving things obvious and meaningful names, this is > the Heap-Only Tuple (HOT) Optimization. This optimization is possible when:" Sorry, I don't like the above since it isn't precise and the "In a case of giving things obvious and meaningful names" seems odd. -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com Indecision is a decision. Inaction is an action. Mark Batterson
Вложения
On Fri, Jul 22, 2022 at 10:05:04AM -0700, Peter Geoghegan wrote: > On Fri, Jul 22, 2022 at 8:09 AM Jonathan S. Katz <jkatz@postgresql.org> wrote: > > Thanks! This is great. Probably the most concise and clear explanation I > > have seen for HOT, which is exactly what we need for the docs :) > > I'm delighted that this is finally happening! > > > I think we should expand on this and explain how adjusting "fillfactor" > > will affect this. I think that may change the final sentence too. > > Definitely -- this is by far the most important reason to tune heap > fill factor, which can make a big difference. There should be a link > from the fill factor docs to the new HOT section, at a minimum. Done in the attached patch, and updated URL: https://momjian.us/tmp/pgsql/storage-hot.html > Another thing: would be good to point out how to monitor the number of > HOT updates vs non-HOT updates using views like pg_stat_user_tables. I added a sentence about that, good idea. > One minor correction: Opportunistic pruning isn't limited to heap-only > tuples -- "HOT pruning" is actually a bit of a misnomer, that somehow > caught on in the Postgres community. Opportunistic pruning can and > will happen even with non-HOT updates -- the only difference on the > heap page level is that 4 byte LP_DEAD line pointers will accumulate > over time, which only VACUUM is able to clean up (we need LP_DEAD > items to stick around until then as tombstones, so that index scans > don't ever get confused). But pruning will still be able to free most > of the space on the heap page, in almost the same way. I see your point. I changed the benefits from pruning to no new page item identifiers. > There is a section about bottom-up index deletion in the docs (in the > B-Tree internals chapter) that already references HOT. It should link > to this new chapter now, I think. Already done in the attached patch, I think. -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com Indecision is a decision. Inaction is an action. Mark Batterson
Вложения
On Fri, Jul 22, 2022 at 01:07:41PM -0400, Bruce Momjian wrote: > > > You can increase the change of HOT updates by using non-default table > > fillfactor settings. > > > > I think we should expand on this and explain how adjusting "fillfactor" will > > affect this. I think that may change the final sentence too. > > Okay, done in the attached patch. I have improved the wording of the last paragraph in this patch. -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com Indecision is a decision. Inaction is an action. Mark Batterson
Вложения
On Fri, Jul 22, 2022 at 2:11 PM Bruce Momjian <bruce@momjian.us> wrote: > I have improved the wording of the last paragraph in this patch. I think that it would be worth prominently explaining where heap-only tuples get their name from: it comes from the fact there are (by definition) no entries for a heap-only tuple in any index, ever. Indexes are nevertheless capable of locating heap-only tuples during index scans, by dealing with a little additional indirection: they must traverse groups of related tuple versions, all for the same logical row that was HOT updated one or more times -- this group of related tuples is called a HOT chain. This seems like a useful thing to emphasize because it places the emphasis on what *doesn't* happen. Mostly what doesn't happen in indexes. New item identifiers actually *are* needed for heap-only tuples (perhaps we could get away with it, but we don't). However, that doesn't really matter too much in practice. Heap-only tuples can still have their line pointers set to LP_UNUSED directly during pruning, without having to be set to LP_DEAD for a time first (a situation which VACUUM alone can correct by setting the LP_DEAD items to LP_UNUSED during its second heap pass). So heap-only tuples "skip the step" where they have to become LP_DEAD stubs/tombstones. Which is possible precisely because indexes don't need to be considered (they're "heap-only"). I agree that pruning should be discussed here, though -- I wouldn't go as far as treating pruning as 100% unrelated to HOT. Perhaps something along the lines of this works: "It is possible for opportunistic pruning to completely remove all bloat caused by HOT updates (bloat from HOT chains), without leaving any residual garbage that only VACUUM is capable of cleaning up. Pruning a page affected by non-HOT updates or deletes is somewhat less effective, though, because small tombstone items (dead item identifiers) must remain until such time as VACUUM can verify that no remaining index tuples reference the items." Again, the emphasis is on what *doesn't* have to happen because indexes aren't making life hard for us. From the point of view of indexes, ignorance is bliss. The really nice important point about pruning and HOT is that it becomes possible (with care from the DBA and application) to practically eliminate the role of VACUUM. We may not even require a little help from VACUUM, under ideal conditions. -- Peter Geoghegan
On Fri, Jul 22, 2022 at 03:33:20PM -0700, Peter Geoghegan wrote: > On Fri, Jul 22, 2022 at 2:11 PM Bruce Momjian <bruce@momjian.us> wrote: > > I have improved the wording of the last paragraph in this patch. > > I think that it would be worth prominently explaining where heap-only > tuples get their name from: it comes from the fact there are (by > definition) no entries for a heap-only tuple in any index, ever. > Indexes are nevertheless capable of locating heap-only tuples during > index scans, by dealing with a little additional indirection: they > must traverse groups of related tuple versions, all for the same > logical row that was HOT updated one or more times -- this group of > related tuples is called a HOT chain. > > This seems like a useful thing to emphasize because it places the > emphasis on what *doesn't* happen. Mostly what doesn't happen in > indexes. > > New item identifiers actually *are* needed for heap-only tuples > (perhaps we could get away with it, but we don't). However, that > doesn't really matter too much in practice. Heap-only tuples can still > have their line pointers set to LP_UNUSED directly during pruning, > without having to be set to LP_DEAD for a time first (a situation > which VACUUM alone can correct by setting the LP_DEAD items to > LP_UNUSED during its second heap pass). > > So heap-only tuples "skip the step" where they have to become LP_DEAD > stubs/tombstones. Which is possible precisely because indexes don't > need to be considered (they're "heap-only"). Good points. I have updated the attached patch and URL to mention that HOT rows are _completely_ removed, and why that is possible, and I clarified the page item identifier mention. -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com Indecision is a decision. Inaction is an action. Mark Batterson
Вложения
On Sat, Jul 23, 2022 at 8:51 AM Bruce Momjian <bruce@momjian.us> wrote: > Good points. I have updated the attached patch and URL to mention that > HOT rows are _completely_ removed, and why that is possible, and I > clarified the page item identifier mention. I think that this version looks very good, but I do have some minor notes: * You wrote "Specifically, updates cause additional rows to be added to tables." Perhaps this could be rephrased: "Specifically, updates add new physical tuples to tables to represent each new version." I think that the term "row" should only refer to the simple/abstract idea of a row from a table, while the term tuple should be preferred when referring to a physical embodiment of a row, like one version of a row. Perhaps it's worth following that convention across the board here (not just in this sentence that I have highlighted). * You wrote "This can also require new index entries for each updated row, and removal of old versions of rows can be expensive" I believe that the operative word in this sentence (which appears in the first paragraph) is "can". I think that it would be good to go just a bit further with that. Maybe add another sentence immediately afterwards that conveys "and now we're going to discuss when and how new versions from updates can sometimes avoid the need for a new round of index entries". * You wrote "New index entries are not needed to represent updated rows" It seems to me that this undersells the key benefit. You could perhaps add another sentence. Something like: "This avoids the immediate cost of adding new successor versions to each and every index, and avoids the cost of removing the obsolete versions from each and every index later on." * You refer to opportunistic pruning as something that happens "during normal operation", but that doesn't seem to get the idea of "opportunistic" across. It seems like it would be worth writing a sentence or two more on this, just to get that aspect across. Opportunistic cleanup occurs when a query happens to notice that a heap page that it had to read as part of query processing needed to be cleaned up in passing. We do it there and then because it happens to be relatively cheap and convenient to do it that way. That sort of thing. Overall, I think that this is suitable to commit, and I don't want to make too much of a fuss. It's great that we're doing this. Thanks -- Peter Geoghegan
On Sat, Jul 23, 2022 at 11:34 AM Peter Geoghegan <pg@bowt.ie> wrote:
On Sat, Jul 23, 2022 at 8:51 AM Bruce Momjian <bruce@momjian.us> wrote:
> Good points. I have updated the attached patch and URL to mention that
> HOT rows are _completely_ removed, and why that is possible, and I
> clarified the page item identifier mention.
I think that this version looks very good, but I do have some minor notes:
* You wrote "Specifically, updates cause additional rows to be added to tables."
Perhaps this could be rephrased: "Specifically, updates add new
physical tuples to tables to represent each new version."
I think that the term "row" should only refer to the simple/abstract
idea of a row from a table, while the term tuple should be preferred
when referring to a physical embodiment of a row, like one version of
a row. Perhaps it's worth following that convention across the board
here (not just in this sentence that I have highlighted).
I concur, suggesting the following:
"Specifically, updates result in multiple rows versions (tuples) existing on the table."
"There is sufficient free space on the page containing the old tuple for the updated tuple."
"Old tuples can be completely removed..."
Overall, I think that this is suitable to commit, and I don't want to
make too much of a fuss. It's great that we're doing this.
Agreed. The other suggestion listed are not clear-cut winners in my mind.
The following, though, seems to just come out of nowhere. It would be better setup as a "(See <link> for why this is possible.)" instead of dropping "page item identifiers" on the reader.
+ This removal is possible because indexes
+ do not reference their <link linkend="storage-page-layout">page
+ item identifiers</link>.
+ do not reference their <link linkend="storage-page-layout">page
+ item identifiers</link>.
As a related thought, this has done a great job of being usable for a DBA operating at a high-level of system knowledge and interaction. I don't think burying it in storage.sgml is desirable, Maybe "Performance Tips" under "Avoid Unnecessary Indexes" (yes, a bit of a stretch, but nothing else seems to fit better, except maybe in concurrency control since we are discussing overcoming the limitation of our concurrency control choice.
Summary paragraph:
"can only happen if" => "can only be created if"
David J.
On Sat, Jul 23, 2022 at 11:33:40AM -0700, Peter Geoghegan wrote: > On Sat, Jul 23, 2022 at 8:51 AM Bruce Momjian <bruce@momjian.us> wrote: > > Good points. I have updated the attached patch and URL to mention that > > HOT rows are _completely_ removed, and why that is possible, and I > > clarified the page item identifier mention. > > I think that this version looks very good, but I do have some minor notes: > > * You wrote "Specifically, updates cause additional rows to be added to tables." > > Perhaps this could be rephrased: "Specifically, updates add new > physical tuples to tables to represent each new version." Uh, that seems more confusing than what I have. I also considered "tuples", but if you are saying "old version of a row", you are taking about an old version of a logical row, not an old version of a physical tuple, really. > I think that the term "row" should only refer to the simple/abstract > idea of a row from a table, while the term tuple should be preferred > when referring to a physical embodiment of a row, like one version of > a row. Perhaps it's worth following that convention across the board > here (not just in this sentence that I have highlighted). Yes, if we were talking about tuples unrelated to the versions of the rows they represent, then yes, it would make sense. > * You wrote "This can also require new index entries for each updated > row, and removal of old versions of rows can be expensive" > > I believe that the operative word in this sentence (which appears in > the first paragraph) is "can". I think that it would be good to go > just a bit further with that. Maybe add another sentence immediately > afterwards that conveys "and now we're going to discuss when and how > new versions from updates can sometimes avoid the need for a new round > of index entries". I ended up adding index cleanup further up in the text --- please see my patch in the next email I send in this thread. > > * You wrote "New index entries are not needed to represent updated rows" > > It seems to me that this undersells the key benefit. You could perhaps > add another sentence. Something like: "This avoids the immediate cost > of adding new successor versions to each and every index, and avoids > the cost of removing the obsolete versions from each and every index > later on." Same. > > * You refer to opportunistic pruning as something that happens "during > normal operation", but that doesn't seem to get the idea of > "opportunistic" across. > > It seems like it would be worth writing a sentence or two more on > this, just to get that aspect across. Opportunistic cleanup occurs > when a query happens to notice that a heap page that it had to read as > part of query processing needed to be cleaned up in passing. We do it > there and then because it happens to be relatively cheap and > convenient to do it that way. That sort of thing. Yes, we need that, but not in this section --- I would like to it though. -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com Indecision is a decision. Inaction is an action. Mark Batterson
On Sat, Jul 23, 2022 at 12:12:38PM -0700, David G. Johnston wrote: > I concur, suggesting the following: > > "Specifically, updates result in multiple rows versions (tuples) existing on > the table." See my reply to Peter G. about tuples. > "There is sufficient free space on the page containing the old tuple for the > updated tuple." Same. > "Old tuples can be completely removed..." It is really old row versions, not old tuples since tuples, in my mind, don't have a chain of versions. There are cannot-be-viewed-by-anyone tuples, but that is not the point here. > Overall, I think that this is suitable to commit, and I don't want to > make too much of a fuss. It's great that we're doing this. > > > > Agreed. The other suggestion listed are not clear-cut winners in my mind. > > The following, though, seems to just come out of nowhere. It would be better > setup as a "(See <link> for why this is possible.)" instead of dropping "page > item identifiers" on the reader. > > + This removal is possible because indexes > + do not reference their <link linkend="storage-page-layout">page > + item identifiers</link>. I added parentheses around that. > > As a related thought, this has done a great job of being usable for a DBA > operating at a high-level of system knowledge and interaction. I don't think > burying it in storage.sgml is desirable, Maybe "Performance Tips" under "Avoid > Unnecessary Indexes" (yes, a bit of a stretch, but nothing else seems to fit > better, except maybe in concurrency control since we are discussing overcoming > the limitation of our concurrency control choice. Uh, not sure. Anyone else have an opinion? > Summary paragraph: > "can only happen if" => "can only be created if" Yes, good point. Updated patch attached. -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com Indecision is a decision. Inaction is an action. Mark Batterson
Вложения
On Tue, Jul 26, 2022 at 07:29:10PM -0400, Bruce Momjian wrote: > > Summary paragraph: > > "can only happen if" => "can only be created if" > > Yes, good point. > > Updated patch attached. I applied this patch back to PG 11. (PG 10 had too many conflicts.) I also added a mention about how indexes can prevent HOT to the index introduction section; applied patch of that addition attached. -- Bruce Momjian <bruce@momjian.us> https://momjian.us EDB https://enterprisedb.com Indecision is a decision. Inaction is an action. Mark Batterson