Обсуждение: doc: New cumulative stats subsystem obsoletes comment in maintenance.sgml

Поиск
Список
Период
Сортировка

doc: New cumulative stats subsystem obsoletes comment in maintenance.sgml

От
"David G. Johnston"
Дата:
Hackers,

The new cumulative stats subsystem no longer has a "lost under heavy load" problem so that parenthetical should go (or at least be modified).

These stats can be reset so some discussion about how the system uses them given that possibility seems like it would be good to add here.  I'm not sure what that should look like though.

David J.


diff --git a/doc/src/sgml/maintenance.sgml b/doc/src/sgml/maintenance.sgml
index 04a04e0e5f..360807c8f9 100644
--- a/doc/src/sgml/maintenance.sgml
+++ b/doc/src/sgml/maintenance.sgml
@@ -652,9 +652,8 @@ vacuum insert threshold = vacuum base insert threshold + vacuum insert scale fac
     tuples to be frozen by earlier vacuums.  The number of obsolete tuples and
     the number of inserted tuples are obtained from the cumulative statistics system;
     it is a semi-accurate count updated by each <command>UPDATE</command>,
-    <command>DELETE</command> and <command>INSERT</command> operation.  (It is
-    only semi-accurate because some information might be lost under heavy
-    load.)  If the <structfield>relfrozenxid</structfield> value of the table
+    <command>DELETE</command> and <command>INSERT</command> operation.
+    If the <structfield>relfrozenxid</structfield> value of the table
     is more than <varname>vacuum_freeze_table_age</varname> transactions old,
     an aggressive vacuum is performed to freeze old tuples and advance
     <structfield>relfrozenxid</structfield>; otherwise, only pages that have been modified

Re: doc: New cumulative stats subsystem obsoletes comment in maintenance.sgml

От
Bruce Momjian
Дата:
On Wed, Apr 20, 2022 at 04:40:44PM -0700, David G. Johnston wrote:
> Hackers,
> 
> The new cumulative stats subsystem no longer has a "lost under heavy load"
> problem so that parenthetical should go (or at least be modified).
> 
> These stats can be reset so some discussion about how the system uses them
> given that possibility seems like it would be good to add here.  I'm not sure
> what that should look like though.
> 
> diff --git a/doc/src/sgml/maintenance.sgml b/doc/src/sgml/maintenance.sgml
> index 04a04e0e5f..360807c8f9 100644
> --- a/doc/src/sgml/maintenance.sgml
> +++ b/doc/src/sgml/maintenance.sgml
> @@ -652,9 +652,8 @@ vacuum insert threshold = vacuum base insert threshold +
> vacuum insert scale fac
>      tuples to be frozen by earlier vacuums.  The number of obsolete tuples and
>      the number of inserted tuples are obtained from the cumulative statistics
> system;
>      it is a semi-accurate count updated by each <command>UPDATE</command>,
> -    <command>DELETE</command> and <command>INSERT</command> operation.  (It is
> -    only semi-accurate because some information might be lost under heavy
> -    load.)  If the <structfield>relfrozenxid</structfield> value of the table
> +    <command>DELETE</command> and <command>INSERT</command> operation.
> +    If the <structfield>relfrozenxid</structfield> value of the table
>      is more than <varname>vacuum_freeze_table_age</varname> transactions old,
>      an aggressive vacuum is performed to freeze old tuples and advance
>      <structfield>relfrozenxid</structfield>; otherwise, only pages that have
> been modified

Yes, I agree and plan to apply this patch soon.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Indecision is a decision.  Inaction is an action.  Mark Batterson




Re: doc: New cumulative stats subsystem obsoletes comment in maintenance.sgml

От
Andres Freund
Дата:
Hi,

I had missed David's original email on this topic...

On 2022-07-14 18:58:09 -0400, Bruce Momjian wrote:
> On Wed, Apr 20, 2022 at 04:40:44PM -0700, David G. Johnston wrote:
> > The new cumulative stats subsystem no longer has a "lost under heavy load"
> > problem so that parenthetical should go (or at least be modified).
> > 
> > These stats can be reset so some discussion about how the system uses them
> > given that possibility seems like it would be good to add here.  I'm not sure
> > what that should look like though.
> > 
> > diff --git a/doc/src/sgml/maintenance.sgml b/doc/src/sgml/maintenance.sgml
> > index 04a04e0e5f..360807c8f9 100644
> > --- a/doc/src/sgml/maintenance.sgml
> > +++ b/doc/src/sgml/maintenance.sgml
> > @@ -652,9 +652,8 @@ vacuum insert threshold = vacuum base insert threshold +
> > vacuum insert scale fac
> >      tuples to be frozen by earlier vacuums.  The number of obsolete tuples and
> >      the number of inserted tuples are obtained from the cumulative statistics
> > system;
> >      it is a semi-accurate count updated by each <command>UPDATE</command>,
> > -    <command>DELETE</command> and <command>INSERT</command> operation.  (It is
> > -    only semi-accurate because some information might be lost under heavy
> > -    load.)  If the <structfield>relfrozenxid</structfield> value of the table
> > +    <command>DELETE</command> and <command>INSERT</command> operation.
> > +    If the <structfield>relfrozenxid</structfield> value of the table
> >      is more than <varname>vacuum_freeze_table_age</varname> transactions old,
> >      an aggressive vacuum is performed to freeze old tuples and advance
> >      <structfield>relfrozenxid</structfield>; otherwise, only pages that have
> > been modified
> 
> Yes, I agree and plan to apply this patch soon.

It might make sense to still say semi-accurate, but adjust the explanation to
say that stats reporting is not instantaneous?

Greetings,

Andres Freund



Re: doc: New cumulative stats subsystem obsoletes comment in maintenance.sgml

От
"David G. Johnston"
Дата:
On Thu, Jul 14, 2022 at 4:31 PM Andres Freund <andres@anarazel.de> wrote:
Hi,

I had missed David's original email on this topic...

On 2022-07-14 18:58:09 -0400, Bruce Momjian wrote:
> On Wed, Apr 20, 2022 at 04:40:44PM -0700, David G. Johnston wrote:
> > The new cumulative stats subsystem no longer has a "lost under heavy load"
> > problem so that parenthetical should go (or at least be modified).
> >
> > These stats can be reset so some discussion about how the system uses them
> > given that possibility seems like it would be good to add here.  I'm not sure
> > what that should look like though.
> >
> > diff --git a/doc/src/sgml/maintenance.sgml b/doc/src/sgml/maintenance.sgml
> > index 04a04e0e5f..360807c8f9 100644
> > --- a/doc/src/sgml/maintenance.sgml
> > +++ b/doc/src/sgml/maintenance.sgml
> > @@ -652,9 +652,8 @@ vacuum insert threshold = vacuum base insert threshold +
> > vacuum insert scale fac
> >      tuples to be frozen by earlier vacuums.  The number of obsolete tuples and
> >      the number of inserted tuples are obtained from the cumulative statistics
> > system;
> >      it is a semi-accurate count updated by each <command>UPDATE</command>,
> > -    <command>DELETE</command> and <command>INSERT</command> operation.  (It is
> > -    only semi-accurate because some information might be lost under heavy
> > -    load.)  If the <structfield>relfrozenxid</structfield> value of the table
> > +    <command>DELETE</command> and <command>INSERT</command> operation.
> > +    If the <structfield>relfrozenxid</structfield> value of the table
> >      is more than <varname>vacuum_freeze_table_age</varname> transactions old,
> >      an aggressive vacuum is performed to freeze old tuples and advance
> >      <structfield>relfrozenxid</structfield>; otherwise, only pages that have
> > been modified
>
> Yes, I agree and plan to apply this patch soon.

It might make sense to still say semi-accurate, but adjust the explanation to
say that stats reporting is not instantaneous?


Unless that delay manifests in executing an UPDATE in a session then looking at these views in the same session and not seeing that update reflected I wouldn't mention it.  Concurrency aspects are reasonably expected here.  But if we do want to mention it maybe:

 "...it is an eventually-consistent count updated by..."

David J.

Re: doc: New cumulative stats subsystem obsoletes comment in maintenance.sgml

От
Andres Freund
Дата:
Hi,

On 2022-07-18 19:47:39 -0700, David G. Johnston wrote:
> On Thu, Jul 14, 2022 at 4:31 PM Andres Freund <andres@anarazel.de> wrote:
> > It might make sense to still say semi-accurate, but adjust the explanation
> > to
> > say that stats reporting is not instantaneous?
> >
> >
> Unless that delay manifests in executing an UPDATE in a session then
> looking at these views in the same session and not seeing that update
> reflected I wouldn't mention it.

Depending on which stats you're looking at, yes, that could totally happen. I
don't think the issue is not seeing changes from the current transaction
though - it's that *after* commit you might not see them for a while (the're
transmitted not more than once a second, and can be delayed up to 60s if
there's contention).

Greetings,

Andres Freund



Re: doc: New cumulative stats subsystem obsoletes comment in maintenance.sgml

От
Bruce Momjian
Дата:
On Mon, Jul 18, 2022 at 08:04:12PM -0700, Andres Freund wrote:
> Hi,
> 
> On 2022-07-18 19:47:39 -0700, David G. Johnston wrote:
> > On Thu, Jul 14, 2022 at 4:31 PM Andres Freund <andres@anarazel.de> wrote:
> > > It might make sense to still say semi-accurate, but adjust the explanation
> > > to
> > > say that stats reporting is not instantaneous?
> > >
> > >
> > Unless that delay manifests in executing an UPDATE in a session then
> > looking at these views in the same session and not seeing that update
> > reflected I wouldn't mention it.
> 
> Depending on which stats you're looking at, yes, that could totally happen. I
> don't think the issue is not seeing changes from the current transaction
> though - it's that *after* commit you might not see them for a while (the're
> transmitted not more than once a second, and can be delayed up to 60s if
> there's contention).

So the docs don't need any changes, I assume.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Indecision is a decision.  Inaction is an action.  Mark Batterson




Re: doc: New cumulative stats subsystem obsoletes comment in maintenance.sgml

От
"David G. Johnston"
Дата:
On Fri, Aug 12, 2022 at 12:48 PM Bruce Momjian <bruce@momjian.us> wrote:
On Mon, Jul 18, 2022 at 08:04:12PM -0700, Andres Freund wrote:
> Hi,
>
> On 2022-07-18 19:47:39 -0700, David G. Johnston wrote:
> > On Thu, Jul 14, 2022 at 4:31 PM Andres Freund <andres@anarazel.de> wrote:
> > > It might make sense to still say semi-accurate, but adjust the explanation
> > > to
> > > say that stats reporting is not instantaneous?
> > >
> > >
> > Unless that delay manifests in executing an UPDATE in a session then
> > looking at these views in the same session and not seeing that update
> > reflected I wouldn't mention it.
>
> Depending on which stats you're looking at, yes, that could totally happen. I
> don't think the issue is not seeing changes from the current transaction
> though - it's that *after* commit you might not see them for a while (the're
> transmitted not more than once a second, and can be delayed up to 60s if
> there's contention).

So the docs don't need any changes, I assume.


I dislike using the word accurate here now, it will be accurate, but we don't promise perfect timeliness.  So it needs to change:

 diff --git a/doc/src/sgml/maintenance.sgml b/doc/src/sgml/maintenance.sgml
index 04a04e0e5f..360807c8f9 100644
--- a/doc/src/sgml/maintenance.sgml
+++ b/doc/src/sgml/maintenance.sgml
@@ -652,9 +652,8 @@ vacuum insert threshold = vacuum base insert threshold + vacuum insert scale fac
     tuples to be frozen by earlier vacuums.  The number of obsolete tuples and
     the number of inserted tuples are obtained from the cumulative statistics system;
     it is a semi-accurate count updated by each <command>UPDATE</command>,
-    <command>DELETE</command> and <command>INSERT</command> operation.  (It is
-    only semi-accurate because some information might be lost under heavy
-    load.)  If the <structfield>relfrozenxid</structfield> value of the table
+    <command>DELETE</command> and <command>INSERT</command> operation.
+    If the <structfield>relfrozenxid</structfield> value of the table
     is more than <varname>vacuum_freeze_table_age</varname> transactions old,
     an aggressive vacuum is performed to freeze old tuples and advance
     <structfield>relfrozenxid</structfield>; otherwise, only pages that have been modified

However, also replace the remaining instance of "a semi-accurate count" with "an eventually-consistent count".

...it is an eventually-consistent count updated by each UPDATE, DELETE, and INSERT operation.

David J.


Re: doc: New cumulative stats subsystem obsoletes comment in maintenance.sgml

От
Bruce Momjian
Дата:
On Fri, Aug 12, 2022 at 12:58:28PM -0700, David G. Johnston wrote:
> I dislike using the word accurate here now, it will be accurate, but we don't
> promise perfect timeliness.  So it needs to change:
> 
>  diff --git a/doc/src/sgml/maintenance.sgml b/doc/src/sgml/maintenance.sgml
> index 04a04e0e5f..360807c8f9 100644
> --- a/doc/src/sgml/maintenance.sgml
> +++ b/doc/src/sgml/maintenance.sgml
> @@ -652,9 +652,8 @@ vacuum insert threshold = vacuum base insert threshold +
> vacuum insert scale fac
>      tuples to be frozen by earlier vacuums.  The number of obsolete tuples and
>      the number of inserted tuples are obtained from the cumulative statistics
> system;
>      it is a semi-accurate count updated by each <command>UPDATE</command>,
> -    <command>DELETE</command> and <command>INSERT</command> operation.  (It is
> -    only semi-accurate because some information might be lost under heavy
> -    load.)  If the <structfield>relfrozenxid</structfield> value of the table
> +    <command>DELETE</command> and <command>INSERT</command> operation.
> +    If the <structfield>relfrozenxid</structfield> value of the table
>      is more than <varname>vacuum_freeze_table_age</varname> transactions old,
>      an aggressive vacuum is performed to freeze old tuples and advance
>      <structfield>relfrozenxid</structfield>; otherwise, only pages that have
> been modified

Done in master.

> However, also replace the remaining instance of "a semi-accurate count" with
> "an eventually-consistent count".

> ...it is an eventually-consistent count updated by each UPDATE, DELETE, and
> INSERT operation.

Also done.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Only you can decide what is important to you.