Обсуждение: First-draft release notes for back-branch releases
I've made a pass over the commit log up to now, and prepared draft release note entries for everything that seemed worth documenting; see https://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=65a82a7649860f8010db581a0d1f12aa92f5969b As I usually do, I dropped all of these into a section for 11.1, although some won't remain there because they don't apply to the v11 branch or were committed in time for 11.0. I'll sort that out on Sunday. Comments please ... regards, tom lane
On 11/2/18 8:14 PM, Tom Lane wrote: > I've made a pass over the commit log up to now, and prepared draft > release note entries for everything that seemed worth documenting; > see > > https://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=65a82a7649860f8010db581a0d1f12aa92f5969b > > As I usually do, I dropped all of these into a section for 11.1, > although some won't remain there because they don't apply to the > v11 branch or were committed in time for 11.0. I'll sort that > out on Sunday. > > Comments please ... Attached is a draft of the press release for review. Please let me know if there are any corrections/suggestions. Thanks! Jonathan
Вложения
On Mon, Nov 05, 2018 at 04:01:59PM -0500, Jonathan S. Katz wrote: > Attached is a draft of the press release for review. Please let me know > if there are any corrections/suggestions. > This update is also the final release for PostgreSQL 9.3, which is now end-of-life and will no longer receive any bug orsecurity fixes. If your environment still uses PostgreSQL 9.3, please make plans to update to a community supported versionsas soon as possible. Please see our [versioning policy](https://www.postgresql.org/support/versioning/) for moreinformation. s/versions/version/ > * Fix how `NULL` values are handled when using `LEFT JOIN` with a parallelized hash joins s/joins/join/ > * Disallows the creation of a new partition from a trigger that is attached to its parent table to prevent a crash; thisbehavior could be modified in a future release > * Fix that prevents crashes in triggers on tables with recently added columns I would replace these two with "Fix crashes in triggers". The details are too esoteric, for the first especially, to list here.
On 2018/11/06 11:25, Noah Misch wrote: > On Mon, Nov 05, 2018 at 04:01:59PM -0500, Jonathan S. Katz wrote: >> Attached is a draft of the press release for review. Please let me know >> if there are any corrections/suggestions. >> * Disallows the creation of a new partition from a trigger that is attached to its parent table to prevent a crash; thisbehavior could be modified in a future release >> * Fix that prevents crashes in triggers on tables with recently added columns > > I would replace these two with "Fix crashes in triggers". The details are too > esoteric, for the first especially, to list here. +1 Thanks, Amit
On 11/5/18 9:58 PM, Amit Langote wrote: > On 2018/11/06 11:25, Noah Misch wrote: >> On Mon, Nov 05, 2018 at 04:01:59PM -0500, Jonathan S. Katz wrote: >>> Attached is a draft of the press release for review. Please let me know >>> if there are any corrections/suggestions. >>> * Disallows the creation of a new partition from a trigger that is attached to its parent table to prevent a crash; thisbehavior could be modified in a future release >>> * Fix that prevents crashes in triggers on tables with recently added columns >> >> I would replace these two with "Fix crashes in triggers". The details are too >> esoteric, for the first especially, to list here. > > +1 Thanks for the feedback. I made the suggested changes upthread as well as the above. Please see attached. Funny enough I had a simplified version of the "trigger crashes" in my first draft, but I thought about expanding it given that future behavior could change. I still feel inclined to elaborate on the "Prevent creation of a partition in a trigger attached to its parent table" but perhaps in future release we will just mention that this behavior is again available. Jonathan
Вложения
Hi, On 2018/11/06 12:49, Jonathan S. Katz wrote: > On 11/5/18 9:58 PM, Amit Langote wrote: >> On 2018/11/06 11:25, Noah Misch wrote: >>> On Mon, Nov 05, 2018 at 04:01:59PM -0500, Jonathan S. Katz wrote: >>>> Attached is a draft of the press release for review. Please let me know >>>> if there are any corrections/suggestions. >>>> * Disallows the creation of a new partition from a trigger that is attached to its parent table to prevent a crash;this behavior could be modified in a future release >>>> * Fix that prevents crashes in triggers on tables with recently added columns >>> >>> I would replace these two with "Fix crashes in triggers". The details are too >>> esoteric, for the first especially, to list here. >> >> +1 > > Thanks for the feedback. I made the suggested changes upthread as well > as the above. Please see attached. > > Funny enough I had a simplified version of the "trigger crashes" in my > first draft, but I thought about expanding it given that future behavior > could change. > > I still feel inclined to elaborate on the "Prevent creation of a > partition in a trigger attached to its parent table" but perhaps in > future release we will just mention that this behavior is again available. Maybe, it's enough that that detail is present in the release note item? Thanks, Amit
Do we need to add anything in the release notes about possible complications in upgrading caused by the 65f39408ee71 / 56535dcdc9e2 issue? If upgrading from the immediately prior point releases to this one, then the shutdown of the previous version might have left an incorrect min recovery point in pg_control, yes? So the error could then occur when starting the new version, even though the bug is now apparently fixed. -- Andrew (irc:RhodiumToad)
Greetings, * Andrew Gierth (andrew@tao11.riddles.org.uk) wrote: > Do we need to add anything in the release notes about possible > complications in upgrading caused by the 65f39408ee71 / 56535dcdc9e2 > issue? > > If upgrading from the immediately prior point releases to this one, then > the shutdown of the previous version might have left an incorrect min > recovery point in pg_control, yes? So the error could then occur when > starting the new version, even though the bug is now apparently fixed. Based on the discussion on IRC and Andrew's comments above, it seems to me like we should really highlight this. Would be nice if we could include some information about what to do if someone is bit by this also... Thanks! Stephen
Вложения
On 11/5/18 11:10 PM, Amit Langote wrote: > Hi, > > On 2018/11/06 12:49, Jonathan S. Katz wrote: >> >> I still feel inclined to elaborate on the "Prevent creation of a >> partition in a trigger attached to its parent table" but perhaps in >> future release we will just mention that this behavior is again available. > > Maybe, it's enough that that detail is present in the release note item? Yes...after sleeping on it, I'll keep the simplified explanation and go with the path above of just mentioning when said functionality is available again. Thanks! Jonathan
Вложения
Stephen Frost <sfrost@snowman.net> writes:
> * Andrew Gierth (andrew@tao11.riddles.org.uk) wrote:
>> Do we need to add anything in the release notes about possible
>> complications in upgrading caused by the 65f39408ee71 / 56535dcdc9e2
>> issue?
>>
>> If upgrading from the immediately prior point releases to this one, then
>> the shutdown of the previous version might have left an incorrect min
>> recovery point in pg_control, yes? So the error could then occur when
>> starting the new version, even though the bug is now apparently fixed.
> Based on the discussion on IRC and Andrew's comments above, it seems to
> me like we should really highlight this. Would be nice if we could
> include some information about what to do if someone is bit by this
> also...
You could be bit by any shutdown of the old code, no, whether it's
part of a pg_upgrade or not? Also, it looks like the bug only affects
standbys (or at least that's what the commit message seems to imply),
which makes it less of a data-loss hazard than it might've been.
regards, tom lane
>>>>> "Tom" == Tom Lane <tgl@sss.pgh.pa.us> writes: Tom> You could be bit by any shutdown of the old code, no, whether it's Tom> part of a pg_upgrade or not? Nothing to do with pg_upgrade, this is likely to bite people just doing an update from the previous minor release. Tom> Also, it looks like the bug only affects standbys (or at least Tom> that's what the commit message seems to imply), which makes it Tom> less of a data-loss hazard than it might've been. The commit message doesn't really show the severity of the problem at all. The problem is this: the updating of minRecoveryPoint in the control file is almost completely broken in the last point releases. It's not an "incorrect calculation" as the commit message says, it's that the bgwriter and checkpointer _do not update the value at all_ except immediately after a checkpoint. That means that it is common to have a situation where the recovery restartpoint is at lsn X, the minRecoveryPoint is at a slightly later lsn Y, but there are on-disk data pages with a _much_ later lsn Z. If such a data page was the subject of a Btree/DELETE record, then any attempt to do recovery will potentially PANIC with a (false) "WAL contains references to invalid pages" error -- if, and only if, at least one client (e.g. a monitoring system) is connected when the record is replayed, which is possible because of the incorrect minRecoveryPoint. The users whose case I was diagnosing on IRC were finding that their monitoring system was sufficient to trigger the problem at least 80% of the time. Consider that the broken minRecoveryPoint can be quite a long way in the past relative to on-disk data pages, so the window of vulnerability isn't necessarily small. So while there _probably_ isn't any data corruption, the standby can get into a state that isn't restartable unless you know to block client connections to it until it has caught up. Rebuilding the standby from the master will work but that may be a significant practical problem if the data is large. -- Andrew (irc:RhodiumToad)
On Tue, Nov 06, 2018 at 11:44:56PM +0000, Andrew Gierth wrote: > The commit message doesn't really show the severity of the problem at > all. I take the blame for that. And my apologies for what it's worth. > The users whose case I was diagnosing on IRC were finding that their > monitoring system was sufficient to trigger the problem at least 80% of > the time. Consider that the broken minRecoveryPoint can be quite a long > way in the past relative to on-disk data pages, so the window of > vulnerability isn't necessarily small. The first report after the last point release on the matter is here, and those folks had exactly the same symptoms with clients aggressively connecting to the standby: https://postgr.es/m/153492341830.1368.3936905691758473953@wrigleys.postgresql.org And this came out pretty quickly. > So while there _probably_ isn't any data corruption, the standby can get > into a state that isn't restartable unless you know to block client > connections to it until it has caught up. Rebuilding the standby from > the master will work but that may be a significant practical problem if > the data is large. The problem would show up if you enforce a crash recovery when restarting the standby, not after when letting it shut down cleanly. Corruptions could actually happen if you try to promote the standby before it reaches the actual recovery LSN when it failed to update minRecoveryPoint after it performed a crash recovery. However this is proving to be a problem only if have a standby do a crash recovery and a promotion immediately afterwards, which does not happen when recovering from a backup as well as the minimum recovery LSN comes from the backup end record, not from the control file. -- Michael
Вложения
>>>>> "Michael" == Michael Paquier <michael@paquier.xyz> writes: >> So while there _probably_ isn't any data corruption, the standby can >> get into a state that isn't restartable unless you know to block >> client connections to it until it has caught up. Rebuilding the >> standby from the master will work but that may be a significant >> practical problem if the data is large. Michael> The problem would show up if you enforce a crash recovery when Michael> restarting the standby, not after when letting it shut down Michael> cleanly. No. The problem shows up on clean shutdowns of the standby too. For example, I just shut down using pg_ctl stop -mfast the standby side of the 9.5.14 cluster I've been testing with. Observe: Database cluster state: shut down in recovery pg_control last modified: Wed Nov 7 01:47:33 2018 Latest checkpoint location: 0/201FCF70 Prior checkpoint location: 0/C541B40 Latest checkpoint's REDO location: 0/138D2038 ... Minimum recovery ending location: 0/201FCFE0 So the minimum recovery location is recorded as 0x201FCFE0, but there are data pages on disk with LSNs as recent as 0x25BAFE80. That's a whole lot of daylight that could contain a btree delete. -- Andrew (irc:RhodiumToad)
On Mon, Nov 05, 2018 at 10:49:30PM -0500, Jonathan S. Katz wrote: > On 11/5/18 9:58 PM, Amit Langote wrote: > > On 2018/11/06 11:25, Noah Misch wrote: > >> On Mon, Nov 05, 2018 at 04:01:59PM -0500, Jonathan S. Katz wrote: > >>> Attached is a draft of the press release for review. Please let me know > >>> if there are any corrections/suggestions. > >>> * Disallows the creation of a new partition from a trigger that is attached to its parent table to prevent a crash;this behavior could be modified in a future release > >>> * Fix that prevents crashes in triggers on tables with recently added columns > >> > >> I would replace these two with "Fix crashes in triggers". The details are too > >> esoteric, for the first especially, to list here. > > > > +1 > > Thanks for the feedback. I made the suggested changes upthread as well > as the above. Please see attached. Looks good. > Funny enough I had a simplified version of the "trigger crashes" in my > first draft, but I thought about expanding it given that future behavior > could change. > > I still feel inclined to elaborate on the "Prevent creation of a > partition in a trigger attached to its parent table" but perhaps in > future release we will just mention that this behavior is again available. It's a gray area.
On 2018/11/07 11:28, Noah Misch wrote: > On Mon, Nov 05, 2018 at 10:49:30PM -0500, Jonathan S. Katz wrote: >> On 11/5/18 9:58 PM, Amit Langote wrote: >>> On 2018/11/06 11:25, Noah Misch wrote: >>>> On Mon, Nov 05, 2018 at 04:01:59PM -0500, Jonathan S. Katz wrote: >>>>> Attached is a draft of the press release for review. Please let me know >>>>> if there are any corrections/suggestions. >>>>> * Disallows the creation of a new partition from a trigger that is attached to its parent table to prevent a crash;this behavior could be modified in a future release >>>>> * Fix that prevents crashes in triggers on tables with recently added columns >>>> >>>> I would replace these two with "Fix crashes in triggers". The details are too >>>> esoteric, for the first especially, to list here. >>> >>> +1 >> >> Thanks for the feedback. I made the suggested changes upthread as well >> as the above. Please see attached. > > Looks good. Sorry, I looked at the updated .md file only now. * Fix several crashes with triggers on partitions One of the fixed bugs involving triggers is not related to partitions [1], so I'm not sure about mentioning the phrase "on partitions" here. Thanks, Amit [1] Fix incorrect expansion of tuples lacking recently-added columns (Andrew Dunstan, Amit Langote) This is known to lead to crashes in triggers on tables with recently-added columns, and could have other symptoms as well.
On 11/6/18 9:35 PM, Amit Langote wrote: > On 2018/11/07 11:28, Noah Misch wrote: >> On Mon, Nov 05, 2018 at 10:49:30PM -0500, Jonathan S. Katz wrote: >>> On 11/5/18 9:58 PM, Amit Langote wrote: >>>> On 2018/11/06 11:25, Noah Misch wrote: >>>>> On Mon, Nov 05, 2018 at 04:01:59PM -0500, Jonathan S. Katz wrote: >>>>>> Attached is a draft of the press release for review. Please let me know >>>>>> if there are any corrections/suggestions. >>>>>> * Disallows the creation of a new partition from a trigger that is attached to its parent table to prevent a crash;this behavior could be modified in a future release >>>>>> * Fix that prevents crashes in triggers on tables with recently added columns >>>>> >>>>> I would replace these two with "Fix crashes in triggers". The details are too >>>>> esoteric, for the first especially, to list here. >>>> >>>> +1 >>> >>> Thanks for the feedback. I made the suggested changes upthread as well >>> as the above. Please see attached. >> >> Looks good. > > Sorry, I looked at the updated .md file only now. > > * Fix several crashes with triggers on partitions > > One of the fixed bugs involving triggers is not related to partitions [1], > so I'm not sure about mentioning the phrase "on partitions" here. Thanks. I left it as "Fix several crashes with triggers" - please see attached. I also added two additional bullet points based on the updates to the upcoming release. Thanks again for the feedback! Jonathan
Вложения
On Wed, Nov 07, 2018 at 02:06:13AM +0000, Andrew Gierth wrote: > So the minimum recovery location is recorded as 0x201FCFE0, but there > are data pages on disk with LSNs as recent as 0x25BAFE80. That's a whole > lot of daylight that could contain a btree delete. Sorry for the time it took. I was just testing what you reported by myself, and indeed I can see that even with a clean shutdown on the standby there could be a gap between minRecoveryPoint and the oldest LSNs used with pages flushed. I have hacked a bit pg_verify_checksums so as it is able to list all the LSNs of relations on disk and compared that with what the control file reported, and there are some mismatches. I can also see that HEAD is able to handle things correctly. How to warn about that in the release notes though? The issue is really to be careful with clients connecting aggressively to a hot standby if the problem shows up. I have designed a test case for that stuff which allows me to reproduce the problem easily, but this requires a small patch on pg_verify_checksums to make sure that all on-disk pages do not have a LSN newer than minRecoveryPoint. This tool could be useful for other purposes like checking the sanity of a data folder. So a TAP test could consist in the following actually: 1) On primary create table aa (a int) with (fillfactor = 10); insert into aa values (generate_series(1,1000)); checkpoint; -- generates post-checkpoint FPWs which standby replays. update aa set a = a + 1; 2) On standby, fill in buffers: select count(*) from aa; 3) Primary again, no FPWs this time: update aa set a = a + 1; 4) Standby, restart point which flushes control file: checkpoint; 5) Shutdown primary with immediate mode. 6) Shutdown standby with fast mode. 7) Check state of control file with on-disk files on standby. -- Michael
Вложения
Hi, On 2018/11/07 13:08, Jonathan S. Katz wrote: > Thanks. I left it as "Fix several crashes with triggers" - please see > attached. Thank you. Just one more comment on the top sentence. * Ensure that if a parent partition has an index created in a tablespace, then all child indexes will be created in that same tablespace I think this should be rewritten to not say "parent partition", because it's a bit confusing to call parent a partition. Also, how about mentioning "automatically created" for child indexes? Like this: * Ensure that automatically created child indexes are created in the same tablespace as the parent partitioned index Thanks, Amit
On 11/7/18 1:59 AM, Amit Langote wrote: > * Ensure that automatically created child indexes are created in the same Thanks, that is much clearer. I have made that update. Thanks again for your help! Jonathan