Re: Enforcing that all WAL has been replayed after restoring from backup

On 10.08.2011 15:34, Simon Riggs wrote:
> On Wed, Aug 10, 2011 at 1:19 PM, Robert Haas<robertmhaas@gmail.com>  wrote:
>> On Wed, Aug 10, 2011 at 6:53 AM, Magnus Hagander<magnus@hagander.net>  wrote:
>>> On Wed, Aug 10, 2011 at 12:44, Heikki Linnakangas
>>> <heikki.linnakangas@enterprisedb.com>  wrote:
>>>> On 10.08.2011 12:29, Magnus Hagander wrote:
>>>>>
>>>>> On Tue, Aug 9, 2011 at 18:07, Tom Lane<tgl@sss.pgh.pa.us>    wrote:
>>>>>>
>>>>>> Heikki Linnakangas<heikki.linnakangas@enterprisedb.com>    writes:
>>>>>>>
>>>>>>> On 09.08.2011 18:20, Alvaro Herrera wrote:
>>>>>>>>
>>>>>>>> How about making the new backup_label field optional?  If absent,
>>>>>>>> assume
>>>>>>>> current behavior.
>>>>>>
>>>>>>> That's how I actually did it in the patch. However, the problem wrt.
>>>>>>> requiring initdb is not the new field in backup_label, it's the new
>>>>>>> field in the control file.
>>>>>>
>>>>>> Yeah.  I think it's too late to be fooling with pg_control for 9.1.
>>>>>> Just fix it in HEAD.
>>>>>
>>>>> Should we add a note to the documentation of pg_basebackup in 9.1
>>>>> telling people to take care about the failure case?
>>>>
>>>> Something like "Note: if you abort the backup before it's finished, the
>>>> backup won't be valid" ? That seems pretty obvious to me, hardly worth
>>>> documenting.
>>>
>>> I meant something more along the line of that it looks ok, but may be corrupted.
>>
>> Yeah.  I'm frankly pretty nervous about shipping 9.1 with this
>> problem, but note that I don't have a better idea.  I'd favor making
>> pg_basebackup emit a warning or maybe even remove the backup if it's
>> aborted midway through.
>
> I don't understand why we need to change pg_control for this?
>
> Why can't we just add a line to backup_label as the first action of
> pg_basebackup and then updated it the last action to show the backup
> set is complete?

Hmm, that's not possible for the 'tar' output, but would work for 'dir' 
output. Another similar idea would be to withhold the control file in 
memory until the end of backup, and append it to the output as last. The 
backup can't be restored until the control file is written out.

That won't protect from more complicated scenarios, like if you take the 
backup without the -x flag, and copy some but not all of the required 
WAL files manually to the pg_xlog directory. But it'd be much better 
than nothing for 9.1.

--   Heikki Linnakangas  EnterpriseDB   http://www.enterprisedb.com