Обсуждение: Replication and fsync

On Thu, Oct 24, 2013 at 10:39 AM,  <maillists0@gmail.com> wrote:
> Newb question.
>
> I'm running 9.1 with a slave using streaming replication. A coworker wants
> to turn off fsync on the master and insists that the slave will still be in
> a usable state if there is a failure on the master. We all know that turning
> off fsync is a bad idea, but I was under the impression that the fsync
> setting would be replicated to the slave, making it useless as a backup in
> this scenario.

No. The setting of fsync in the master is not replicated to the standby.

> Am I wrong? If I'm wrong, is there still danger to the slave
> in this kind of setup?

No, I think.

Regards,

--
Fujii Masao

On 24 October 2013 15:04, Fujii Masao <masao.fujii@gmail.com> wrote:
> On Thu, Oct 24, 2013 at 10:39 AM,  <maillists0@gmail.com> wrote:
>> Am I wrong? If I'm wrong, is there still danger to the slave
>> in this kind of setup?
>
> No, I think.

Corruption due to fsync being off on the master will be replicated to
the slave, or - if corruption is bad enough - replication will fail to
replicate affected records entirely. Of course, turning fsync off is
no guarantee for corruption - it's the other way around: having it on
guarantees that you don't get corruption (provided that... etc).

You could disable replication while fsync is off. I'd verify the data
on the master (by creating a dump, for example) before re-enabling it
again, though.

--
If you can't see the forest for the trees,
Cut the trees and you'll see there is no forest.

On Oct 24, 2013, at 18:10, maillists0@gmail.com wrote:

> Thank you for the answers. I'm still confused. If fsync is not replicated to the slave, then how is replication
affectedby a corrupt master? If the master dies and there's a commit recorded in the wal log that didn't actually
happen,wouldn't the slave still be expected to be in a sane state, with the wal logs accurately reflecting what's on
disk? 
>
> Maybe I just don't understand streaming replication enough. The docs seem to say that synchronous commits mean that
theslave also has to verify a write before a transaction is considered complete. How does fsync affect the way/order in
whichstatements are sent to the slave for replication? 

What you're missing is that the master will be replicating corrupt data. That is, _if_ it gets corrupted of course.
But, data corruption in a database has a tendency to go unnoticed for a while.

A corrupted master doesn't necessarily break down immediately - in fact, it can remain running for quite a while as
longas the corruption doesn't break stuff in the wrong places or as long as the corrupted records don't get fetched. 
Until that time, corruption is just blocks of data on disk, which quite possibly end up being replicated to the slave.

Alban Hertroys
--
If you can't see the forest for the trees,
cut the trees and you'll find there is no forest.

Dear

According to manual, when you set "synchronous_commit" to on, the transaction commits will wait until master and slave flush the commit record of transaction to the physical storage, so I think even if turn off the fsync on master is safe for data consistency and data will not be lost if slave physical storage is not damaged.

If "synchronous_commit" is set to remote_write, the transaction commits will wait until slave to write the data to os. Data will lose if master crashed and slave os crashed. But acording to WAL documents it will not crash the data consistency on slave if slave fsync not off.

Otherwise fsync off on master may will result in losing data when master crashes, but still keep consistency on the slave if the slave is not crashed or slave's fsync is on .

See follow:
http://www.postgresql.org/docs/current/static/runtime-config-wal.html#GUC-FSYNC fsync and synchronous_commit
http://www.postgresql.org/docs/current/static/wal-intro.html

DDT wrote:
> According to manual, when you set "synchronous_commit" to on, the transaction commits will wait until
> master and slave flush the commit record of transaction to the physical storage, so I think even if
> turn off the fsync on master is safe for data consistency and data will not be lost if slave physical
> storage is not damaged.

I don't think that this is safe.

What if the master crashes and becomes corrupted as a consequence?

It will start sending corrupted data to the slave, which will
replay it, thus becoming corrupted itself.

Yours,
Laurenz Albe

Hi,

On 24.10.2013 23:18, Alban Hertroys wrote:
> On Oct 24, 2013, at 18:10, maillists0@gmail.com wrote:
>
>> Thank you for the answers. I'm still confused. If fsync is not
>> replicated to the slave, then how is replication affected by a
>> corrupt master? If the master dies and there's a commit recorded in
>> the wal log that didn't actually happen, wouldn't the slave still
>> be expected to be in a sane state, with the wal logs accurately
>> reflecting what's on disk?
>>
>> Maybe I just don't understand streaming replication enough. The
>> docs seem to say that synchronous commits mean that the slave also
>> has to verify a write before a transaction is considered complete.
>> How does fsync affect the way/order in which statements are sent to
>> the slave for replication?
>
> What you're missing is that the master will be replicating corrupt
> data. That is, _if_ it gets corrupted of course. But, data corruption
> in a database has a tendency to go unnoticed for a while.

I think it's safe as long as you don't try to reuse the cluster after a
crash (be it due to OS error, power outage, ...). If the primary crashes
for any reasons, you have to start from scratch, otherwise there might
be silent corruption as you've described.

regards
Tomas

Tomas Vondra <tv@fuzzy.cz> wrote:

> I think it's safe as long as you don't try to reuse the cluster
> after a crash (be it due to OS error, power outage, ...). If the
> primary crashes for any reasons, you have to start from scratch,
> otherwise there might be silent corruption as you've described.

I agree.  It seems to me that as long as you never try to start the
old master after a crash of the OS (directly or because of hardware
or VM failure), the standby should be usable without risk of
corruption.  As soon as you start the old master after such a crash
though, you could be replicating corrupted blocks; you would need
to be very hard-line about never bringing the old master back up.

--
Kevin Grittner
EDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company