Обсуждение: Re: Schedule for migration to pglister

On Mon, Nov 06, 2017 at 10:36:38AM -0500, Stephen Frost wrote:
> Our planned migration schedule is as follows:
> 
> Nov 6 -
>   pgsql-www
> 
> Nov 13 -
>   pgsql-hackers

When each list migrated, its mbox archives stopped receiving new messages:

https://www.postgresql.org/list/pgsql-www/mbox/pgsql-www.201711
https://www.postgresql.org/list/pgsql-hackers/mbox/pgsql-hackers.201711

Noah,

* Noah Misch (noah@leadboat.com) wrote:
> On Mon, Nov 06, 2017 at 10:36:38AM -0500, Stephen Frost wrote:
> > Our planned migration schedule is as follows:
> >
> > Nov 6 -
> >   pgsql-www
> >
> > Nov 13 -
> >   pgsql-hackers
>
> When each list migrated, its mbox archives stopped receiving new messages:

This was actually anticipated, though we had been thinking that the
migration would be faster and so it wouldn't end up being such a long
time that the mbox's wouldn't get updated.

The plan is to replace those mbox's (which are created by mj2 and then
rsync'd over currently) with ones generated from the archives database,
but that's something which will basically happen to the entire site and
all the lists at once.

Further, there will actually be some differences between the
archive-generated mbox's vs. those that came from mj2; in particular,
the 'date' used by mj2 is the 'received' date (from what I can tell)
while the 'date' in the archives database is the 'Date:' header from
when the message is sent by the user.

That said, this will make the mbox's match the actual webpages, which
they don't currently because the webpages are built from the archive
database while the mbox's are from mj2.  In addition, while trying to
cross-compare the two, I think I'm seeing some cases where mj2 is just
outright dropping messages also (for example, I don't see

https://www.postgresql.org/message-id/20170930224424.ud5ilchmclbl5y5n%40alap3.anarazel.de

anywhere in the mj2 mboxes..).

I'm chatting w/ Magnus about this now and I'm not sure exactly when
we'll end up making the change, but my feeling is that we should either
do it now, or on Monday when we migrate the 'user' lists.  Doesn't seem
to me like we should delay it any longer than that.

Thanks!

Stephen

Noah,

* Noah Misch (noah@leadboat.com) wrote:
> On Mon, Nov 06, 2017 at 10:36:38AM -0500, Stephen Frost wrote:
> > Our planned migration schedule is as follows:
> >
> > Nov 6 -
> >   pgsql-www
> >
> > Nov 13 -
> >   pgsql-hackers
>
> When each list migrated, its mbox archives stopped receiving new messages:

This was actually anticipated, though we had been thinking that the
migration would be faster and so it wouldn't end up being such a long
time that the mbox's wouldn't get updated.

The plan is to replace those mbox's (which are created by mj2 and then
rsync'd over currently) with ones generated from the archives database,
but that's something which will basically happen to the entire site and
all the lists at once.

Further, there will actually be some differences between the
archive-generated mbox's vs. those that came from mj2; in particular,
the 'date' used by mj2 is the 'received' date (from what I can tell)
while the 'date' in the archives database is the 'Date:' header from
when the message is sent by the user.

That said, this will make the mbox's match the actual webpages, which
they don't currently because the webpages are built from the archive
database while the mbox's are from mj2.  In addition, while trying to
cross-compare the two, I think I'm seeing some cases where mj2 is just
outright dropping messages also (for example, I don't see

https://www.postgresql.org/message-id/20170930224424.ud5ilchmclbl5y5n%40alap3.anarazel.de

anywhere in the mj2 mboxes..).

I'm chatting w/ Magnus about this now and I'm not sure exactly when
we'll end up making the change, but my feeling is that we should either
do it now, or on Monday when we migrate the 'user' lists.  Doesn't seem
to me like we should delay it any longer than that.

Thanks!

Stephen

On Wed, Nov 15, 2017 at 09:18:31AM -0500, Stephen Frost wrote:
> * Noah Misch (noah@leadboat.com) wrote:
> > When each list migrated, its mbox archives stopped receiving new messages:
> 
> This was actually anticipated, though we had been thinking that the
> migration would be faster and so it wouldn't end up being such a long
> time that the mbox's wouldn't get updated.
> 
> The plan is to replace those mbox's (which are created by mj2 and then
> rsync'd over currently) with ones generated from the archives database,
> but that's something which will basically happen to the entire site and
> all the lists at once.

> I'm chatting w/ Magnus about this now and I'm not sure exactly when
> we'll end up making the change, but my feeling is that we should either
> do it now, or on Monday when we migrate the 'user' lists.  Doesn't seem
> to me like we should delay it any longer than that.

Gotcha.  I, for one, can wait awhile.

On Wed, Nov 15, 2017 at 09:18:31AM -0500, Stephen Frost wrote:
> * Noah Misch (noah@leadboat.com) wrote:
> > When each list migrated, its mbox archives stopped receiving new messages:
> 
> This was actually anticipated, though we had been thinking that the
> migration would be faster and so it wouldn't end up being such a long
> time that the mbox's wouldn't get updated.
> 
> The plan is to replace those mbox's (which are created by mj2 and then
> rsync'd over currently) with ones generated from the archives database,
> but that's something which will basically happen to the entire site and
> all the lists at once.

> I'm chatting w/ Magnus about this now and I'm not sure exactly when
> we'll end up making the change, but my feeling is that we should either
> do it now, or on Monday when we migrate the 'user' lists.  Doesn't seem
> to me like we should delay it any longer than that.

Gotcha.  I, for one, can wait awhile.

Magnus, all,

* Magnus Hagander (magnus@hagander.net) wrote:
> mbox access should now be restored, and as Stephen says, now handled by the
> Date in the message. Let me know if there are any issues with the new ones.
> (They're now generated by the same code as the per-thread ones).

And just fyi, that also means that complete threads end up in each mbox,
so you'll see cases where a message was posted to, eg, -committers
initially but the reply cc'd hackers, bringing the thread over there,
and now the initial email to -committers and all of the subsequent
emails on the thread will be included in the mbox for both -hackers and
-committers.

Hopefully everyone finds that to actually provide a better experience,
but it's a difference to be aware of.

Thanks!

Stephen

Magnus, all,

* Magnus Hagander (magnus@hagander.net) wrote:
> mbox access should now be restored, and as Stephen says, now handled by the
> Date in the message. Let me know if there are any issues with the new ones.
> (They're now generated by the same code as the per-thread ones).

And just fyi, that also means that complete threads end up in each mbox,
so you'll see cases where a message was posted to, eg, -committers
initially but the reply cc'd hackers, bringing the thread over there,
and now the initial email to -committers and all of the subsequent
emails on the thread will be included in the mbox for both -hackers and
-committers.

Hopefully everyone finds that to actually provide a better experience,
but it's a difference to be aware of.

Thanks!

Stephen

On 2017-11-16 15:34, Stephen Frost wrote:
> Magnus, all,
> 
> * Magnus Hagander (magnus@hagander.net) wrote:
>> mbox access should now be restored, and as Stephen says, now handled 
>> by the
>> Date in the message. Let me know if there are any issues with the new 
>> ones.
>> (They're now generated by the same code as the per-thread ones).
> 
> And just fyi, that also means that complete threads end up in each 
> mbox,
> so you'll see cases where a message was posted to, eg, -committers
> initially but the reply cc'd hackers, bringing the thread over there,
> and now the initial email to -committers and all of the subsequent
> emails on the thread will be included in the mbox for both -hackers and
> -committers.
> 
> Hopefully everyone finds that to actually provide a better experience,
> but it's a difference to be aware of.

In theory :), that sounds better for people reading the web archives.

Only one real way to find out "for sure" though, so fingers crossed. :)

+ Justin

On 2017-11-16 15:34, Stephen Frost wrote:
> Magnus, all,
> 
> * Magnus Hagander (magnus@hagander.net) wrote:
>> mbox access should now be restored, and as Stephen says, now handled 
>> by the
>> Date in the message. Let me know if there are any issues with the new 
>> ones.
>> (They're now generated by the same code as the per-thread ones).
> 
> And just fyi, that also means that complete threads end up in each 
> mbox,
> so you'll see cases where a message was posted to, eg, -committers
> initially but the reply cc'd hackers, bringing the thread over there,
> and now the initial email to -committers and all of the subsequent
> emails on the thread will be included in the mbox for both -hackers and
> -committers.
> 
> Hopefully everyone finds that to actually provide a better experience,
> but it's a difference to be aware of.

In theory :), that sounds better for people reading the web archives.

Only one real way to find out "for sure" though, so fingers crossed. :)

+ Justin