Обсуждение: Git migration timeline
Hi! According to the decision at the developer meeting, the migration to git should happen 17-20 Aug. Here's my proposed timeline. This will obviously affect development work some, and since the original timeline called for us having already released 9.0 buy then ;) 1. Tuesday evening, around 19:00 central european time, which is 17:00 GMT or 12:00 EST, I will freeze the current cvs repository. I will do this by disabling committer login on that box, so please note that this will also make it impossible for committers to do a "cvs update" from the authenticated repository. The anonymous repository will of course still be available. 2. Tuesday evening and wednesday morning (european time again, please extrapolate as necessary yourself :P), I will convert the latest cvs version including running all my comparison scripts. (this takes quite a long time). once the conversion is done, I'll snapshot whatever we have by then so we can back out to that. 3. I will then as soon as this is done run the script to remove keywords from active branches. 4. At this point, I will push a copy of this to the postgresql-migration.git project on git.postgresql.org, asking everybody to test. Those of you around, please do so as quickly as you can. It will also be made available to all committers at this time as a read-only repository. 5. Once enough is verified, we'll open the repository up for commits. I expect this to happen sometime on thursday probably, assuming there were no issues. 6. Once things have been running for a few days, we'll replace the postgresql.git project on git.postgresql.org with an automatic push of the repository, renaming the old one to something like postgresql-old.git for a while. 7. As things are done, I'll get the anoncvs mirror moving as soon as possible. But I want to make sure that the git stuff is fully working before we do that. This means the following: * If anybody is working on some specific patch they want to get in soon, please try to do so before tuesday evening european time. * If you are working off the old git tree, you are recommended to do a pull sometime between that and thuersday. That'll make sure you have the latest copy of the *old* git repository. You can then generate a patch off that and submit it or re-apply to your old repository. Note that the new repository will *NOT* be compatible with the old one, so you'll need to do a manual move over. All commits will have new SHA1'sl. * Buildfarm coverage will obviously be weak until BF members have been converted to run off git. Those running directly against anoncvs should have a fair chance to still work on the new one once that's up, but the rsync-the-cvs-repository method will no longer work. Um, I'm sure there's more, but... . Are there any objections to this timeline, or will this work out reasonably well for everybody? -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/
Magnus Hagander <magnus@hagander.net> writes: > According to the decision at the developer meeting, the migration to > git should happen 17-20 Aug. Here's my proposed timeline. This will > obviously affect development work some, and since the original > timeline called for us having already released 9.0 buy then ;) Core is currently talking about exactly when we want to push out 9.0beta5 and/or 9.1alpha1, and the scheduling of the git transition has to wait on that decision. I don't think we're going to be ready for it to begin on Tuesday. Ignoring the exact timing, this plan sounds good except for this bit: > 1. Tuesday evening, around 19:00 central european time, which is 17:00 > GMT or 12:00 EST, I will freeze the current cvs repository. I will do > this by disabling committer login on that box, so please note that > this will also make it impossible for committers to do a "cvs update" > from the authenticated repository. That's not really going to do, especially since it will also lock out "cvs log". I certainly want to do a final update and cvs2cl after the last commits have happened, and I imagine other people will want that too. If there were a way to make the repository read-only that would be nice, but I think what we're going to need to settle for is relying on the committers to not do any more commits after the specified time. I can see the point of wanting to be dead certain the repository isn't changing under you during the data migration. Perhaps we need an agreed window between last call for commits and the actual lock-out. regards, tom lane
* Tom Lane <tgl@sss.pgh.pa.us> [100816 09:58]: > That's not really going to do, especially since it will also lock out > "cvs log". I certainly want to do a final update and cvs2cl after the > last commits have happened, and I imagine other people will want that > too. If there were a way to make the repository read-only that would be > nice, but I think what we're going to need to settle for is relying on > the committers to not do any more commits after the specified time. > > I can see the point of wanting to be dead certain the repository isn't > changing under you during the data migration. Perhaps we need an agreed > window between last call for commits and the actual lock-out. What about 2 minutes long for the repository to be "synced" to wherever he wants to do the conversion from. There isn't any reason for the repository to be locked while he does the conversion/checks, but he does need a guaranteed consistent copy of the repository to work from. I'ld suggest having the repository "locked" for long enough to: 1) Get a consistent rsync/snapshot/tarball of $CVSROOT 2) Send an email to committers saying CVS repository has been "taken", don't commit/tag/branch on it anymore "please" And then unlock it with the expectation that committers will continue in their good stewardship of the CVS repository (in this case meaning not make any changes to it anymore) as they are already doing. a. -- Aidan Van Dyk Create like a god, aidan@highrise.ca command like a king, http://www.highrise.ca/ work like a slave.
On Mon, Aug 16, 2010 at 15:58, Tom Lane <tgl@sss.pgh.pa.us> wrote: > Magnus Hagander <magnus@hagander.net> writes: >> According to the decision at the developer meeting, the migration to >> git should happen 17-20 Aug. Here's my proposed timeline. This will >> obviously affect development work some, and since the original >> timeline called for us having already released 9.0 buy then ;) > > Core is currently talking about exactly when we want to push out > 9.0beta5 and/or 9.1alpha1, and the scheduling of the git transition has > to wait on that decision. I don't think we're going to be ready for it > to begin on Tuesday. Ignoring the exact timing, this plan sounds good > except for this bit: Ok. The sooner you can let me (and others) know, the better. Note that if it gets pushed past the end of this week (for the whole process), it's quite likely that the whole process will be stretched out further. I'll have other committments I need to take care of meanwhile then - this week i've set of a fair amount of dedicated time to work on this. Next week, for example, almost all the work would have to be done durning the evenings (european time) outside of work hours, which obviously makes for a lot less available time. >> 1. Tuesday evening, around 19:00 central european time, which is 17:00 >> GMT or 12:00 EST, I will freeze the current cvs repository. I will do >> this by disabling committer login on that box, so please note that >> this will also make it impossible for committers to do a "cvs update" >> from the authenticated repository. > > That's not really going to do, especially since it will also lock out > "cvs log". I certainly want to do a final update and cvs2cl after the > last commits have happened, and I imagine other people will want that > too. If there were a way to make the repository read-only that would be > nice, but I think what we're going to need to settle for is relying on > the committers to not do any more commits after the specified time. You can still rsync the copy off the anoncvs server (or use the anoncvs one). That one is an exact rsync of the main one, so I don't see why something like "cvs2cl" wouldn't work off that one automatically. I was just assuming every developer was already using an rsynced copy for that kind of work :-) I would like to make the repo readonly if I could, but I don't know a reliable way of doing that with cvs. If someone can enlighten me on one, that's obviously preferred. > I can see the point of wanting to be dead certain the repository isn't > changing under you during the data migration. Perhaps we need an agreed > window between last call for commits and the actual lock-out. To prevent that, I just need to shut it down for 2 minutes to rsync the latest changes off it and onto the new git box. Maybe that's how we should do it. This is a defense against committers forgetting about things and accidentally committing to the repository *after* it's been snapshotted. I guess we can just declare those as ignorant and throw away those commits.. :-) -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/
Magnus Hagander <magnus@hagander.net> writes: > On Mon, Aug 16, 2010 at 15:58, Tom Lane <tgl@sss.pgh.pa.us> wrote: >> I can see the point of wanting to be dead certain the repository isn't >> changing under you during the data migration. �Perhaps we need an agreed >> window between last call for commits and the actual lock-out. > To prevent that, I just need to shut it down for 2 minutes to rsync > the latest changes off it and onto the new git box. Maybe that's how > we should do it. That sounds like a reasonable scheme to me, especially since it leaves the cvs repository functional. Dunno about you, but I want a fallback plan in case this turns into a disaster ... regards, tom lane
On Mon, Aug 16, 2010 at 9:58 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote: >> 1. Tuesday evening, around 19:00 central european time, which is 17:00 >> GMT or 12:00 EST, I will freeze the current cvs repository. I will do >> this by disabling committer login on that box, so please note that >> this will also make it impossible for committers to do a "cvs update" >> from the authenticated repository. > > That's not really going to do, especially since it will also lock out > "cvs log". I certainly want to do a final update and cvs2cl after the > last commits have happened, and I imagine other people will want that > too. If there were a way to make the repository read-only that would be > nice, but I think what we're going to need to settle for is relying on > the committers to not do any more commits after the specified time. > > I can see the point of wanting to be dead certain the repository isn't > changing under you during the data migration. Perhaps we need an agreed > window between last call for commits and the actual lock-out. I would feel a lot more comfortable if the CVS repository was locked down before we start running the migration. Any accidental commits would be a nuisance. However, giving a window between last call for commits and actual lock-out seems reasonable. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise Postgres Company
On Mon, Aug 16, 2010 at 10:15 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote: > Magnus Hagander <magnus@hagander.net> writes: >> On Mon, Aug 16, 2010 at 15:58, Tom Lane <tgl@sss.pgh.pa.us> wrote: >>> I can see the point of wanting to be dead certain the repository isn't >>> changing under you during the data migration. Perhaps we need an agreed >>> window between last call for commits and the actual lock-out. > >> To prevent that, I just need to shut it down for 2 minutes to rsync >> the latest changes off it and onto the new git box. Maybe that's how >> we should do it. > > That sounds like a reasonable scheme to me, especially since it leaves > the cvs repository functional. Dunno about you, but I want a fallback > plan in case this turns into a disaster ... I don't think anybody proposed permanently deleting the CVS repository. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise Postgres Company
On Mon, Aug 16, 2010 at 16:15, Tom Lane <tgl@sss.pgh.pa.us> wrote: > Magnus Hagander <magnus@hagander.net> writes: >> On Mon, Aug 16, 2010 at 15:58, Tom Lane <tgl@sss.pgh.pa.us> wrote: >>> I can see the point of wanting to be dead certain the repository isn't >>> changing under you during the data migration. Perhaps we need an agreed >>> window between last call for commits and the actual lock-out. > >> To prevent that, I just need to shut it down for 2 minutes to rsync >> the latest changes off it and onto the new git box. Maybe that's how >> we should do it. > > That sounds like a reasonable scheme to me, especially since it leaves > the cvs repository functional. Dunno about you, but I want a fallback > plan in case this turns into a disaster ... Oh, I had no intention whatsoever of *removing* it. Just disabling login to it. -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/
On Mon, 2010-08-16 at 16:19 +0200, Magnus Hagander wrote: > On Mon, Aug 16, 2010 at 16:15, Tom Lane <tgl@sss.pgh.pa.us> wrote: > > Magnus Hagander <magnus@hagander.net> writes: > >> On Mon, Aug 16, 2010 at 15:58, Tom Lane <tgl@sss.pgh.pa.us> wrote: > >>> I can see the point of wanting to be dead certain the repository isn't > >>> changing under you during the data migration. Perhaps we need an agreed > >>> window between last call for commits and the actual lock-out. > > > >> To prevent that, I just need to shut it down for 2 minutes to rsync > >> the latest changes off it and onto the new git box. Maybe that's how > >> we should do it. > > > > That sounds like a reasonable scheme to me, especially since it leaves > > the cvs repository functional. Dunno about you, but I want a fallback > > plan in case this turns into a disaster ... > > Oh, I had no intention whatsoever of *removing* it. Just disabling login to it. +1 JD -- PostgreSQL.org Major Contributor Command Prompt, Inc: http://www.commandprompt.com/ - 509.416.6579 Consulting, Training, Support, Custom Development, Engineering http://twitter.com/cmdpromptinc | http://identi.ca/commandprompt
I wrote: > Magnus Hagander <magnus@hagander.net> writes: >> According to the decision at the developer meeting, the migration to >> git should happen 17-20 Aug. Here's my proposed timeline. This will >> obviously affect development work some, and since the original >> timeline called for us having already released 9.0 buy then ;) > Core is currently talking about exactly when we want to push out > 9.0beta5 and/or 9.1alpha1, and the scheduling of the git transition has > to wait on that decision. I don't think we're going to be ready for it > to begin on Tuesday. The current feeling among core seems to be that we should allow the git conversion to proceed according to Magnus' proposed schedule. This would mean that we will *not* be able to wrap either a 9.0 update or 9.1alpha1 releases this week. What we'd probably try to do instead is wrap 9.1alpha1 early next week, as the first attempt to generate tarballs from the git repository, and then wrap 9.0beta5 (or rc1) later in the week. This slips the 9.0 schedule an extra week, but considering that nobody seems to be doing anything about the open 9.0 issues, that was likely to happen anyway. If anybody's really unhappy with this timeline, speak up now. regards, tom lane
On Mon, Aug 16, 2010 at 1:47 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote: > I wrote: >> Magnus Hagander <magnus@hagander.net> writes: >>> According to the decision at the developer meeting, the migration to >>> git should happen 17-20 Aug. Here's my proposed timeline. This will >>> obviously affect development work some, and since the original >>> timeline called for us having already released 9.0 buy then ;) > >> Core is currently talking about exactly when we want to push out >> 9.0beta5 and/or 9.1alpha1, and the scheduling of the git transition has >> to wait on that decision. I don't think we're going to be ready for it >> to begin on Tuesday. > > The current feeling among core seems to be that we should allow the git > conversion to proceed according to Magnus' proposed schedule. This > would mean that we will *not* be able to wrap either a 9.0 update or > 9.1alpha1 releases this week. What we'd probably try to do instead is > wrap 9.1alpha1 early next week, as the first attempt to generate > tarballs from the git repository, and then wrap 9.0beta5 (or rc1) later > in the week. This slips the 9.0 schedule an extra week, but considering > that nobody seems to be doing anything about the open 9.0 issues, that > was likely to happen anyway. > > If anybody's really unhappy with this timeline, speak up now. That sounds good, except for the part about nobody doing anything about the 9.0 open issues. Those issues are: * Backup procedure is wrong? - Nobody's been able to clearly articulate what the problem is, and according to Fujii Masao it's been this way since 8.2. * Walreceiver crashes in AIX - The person who originally reported this problem has been slow to get back to us with the information needed to figure out what is going on. It would be helpful if someone else could test to see if this a general problem with AIX, or something specific to the reporter's installation. Or if someone wants to provide me with a login... * BUG #5595: Documentation is not installs from VPATH build. - I'm not sure this is really a release-blocker, but either way, I have been assuming it's Peter's issue to fix. * trace_recovery_messages inconsistency - This one seems like a simple case of deciding what makes most sense, and doing it. We should definitely fix this before the wrap. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise Postgres Company
Robert Haas <robertmhaas@gmail.com> wrote: > That sounds good, except for the part about nobody doing anything > about the 9.0 open issues. Those issues are: > > [four issues listed] Nobody responded when I asked about this recently, but shouldn't that list include "BUG #5607: memmory leak in ecpg"? We have a patch from Zoltán Böszörményi from before this bug report which seems to address the issue and which Michael Meskes said "Feel free to apply". We don't want to ship 9.0 with known memory leaks, do we? -Kevin
"Kevin Grittner" <Kevin.Grittner@wicourts.gov> writes: > Nobody responded when I asked about this recently, but shouldn't > that list include "BUG #5607: memmory leak in ecpg"? We have a > patch from Zolt�n B�sz�rm�nyi from before this bug report which > seems to address the issue and which Michael Meskes said "Feel free > to apply". > We don't want to ship 9.0 with known memory leaks, do we? Better a memory leak than broken ecpg ;-). Nobody except Michael is terribly comfortable with that code, so we'd all rather wait for him to review and apply the patch. More generally, pre-existing bugs have never been considered release stoppers. At this point what we would block the release for is *new* bugs in 9.0. (An exception to that general rule is pre-existing bugs that would require an initdb to fix; but this one isn't that either.) regards, tom lane
Robert Haas <robertmhaas@gmail.com> writes: > That sounds good, except for the part about nobody doing anything > about the 9.0 open issues. Those issues are: > * Backup procedure is wrong? - Nobody's been able to clearly > articulate what the problem is, and according to Fujii Masao it's been > this way since 8.2. Just because it's old doesn't mean it's not a bug ;-). Normally I would say that a pre-existing documentation problem isn't a release blocker, but in this case that procedure is likely to become of interest to a whole lot more people than it was before, thanks to SR/HS. So we need to understand whether there's a problem and fix it. > * Walreceiver crashes in AIX - The person who originally reported this > problem has been slow to get back to us with the information needed to > figure out what is going on. Yes. If he doesn't provide adequate data, and we can't reproduce it elsewhere, we should not consider this a release blocker. > * BUG #5595: Documentation is not installs from VPATH build. - I'm not > sure this is really a release-blocker, but either way, I have been > assuming it's Peter's issue to fix. I believe this is a regression from prior branches, probably caused by the switch away from distributing the prebuilt docs as sub-tarballs. As such, it's at least a candidate release-blocker. > * trace_recovery_messages inconsistency - This one seems like a simple > case of deciding what makes most sense, and doing it. We should > definitely fix this before the wrap. Agreed, it's a matter of getting some consensus. regards, tom lane
On Mon, 2010-08-16 at 16:19 +0200, Magnus Hagander wrote: > On Mon, Aug 16, 2010 at 16:15, Tom Lane <tgl@sss.pgh.pa.us> wrote: > > Magnus Hagander <magnus@hagander.net> writes: > >> On Mon, Aug 16, 2010 at 15:58, Tom Lane <tgl@sss.pgh.pa.us> wrote: > >>> I can see the point of wanting to be dead certain the repository isn't > >>> changing under you during the data migration. Perhaps we need an agreed > >>> window between last call for commits and the actual lock-out. > > > >> To prevent that, I just need to shut it down for 2 minutes to rsync > >> the latest changes off it and onto the new git box. Maybe that's how > >> we should do it. > > > > That sounds like a reasonable scheme to me, especially since it leaves > > the cvs repository functional. Dunno about you, but I want a fallback > > plan in case this turns into a disaster ... > > Oh, I had no intention whatsoever of *removing* it. Just disabling login to it. +1 JD -- PostgreSQL.org Major Contributor Command Prompt, Inc: http://www.commandprompt.com/ - 509.416.6579 Consulting, Training, Support, Custom Development, Engineering http://twitter.com/cmdpromptinc | http://identi.ca/commandprompt
Magnus Hagander <magnus@hagander.net> writes: > According to the decision at the developer meeting, the migration to > git should happen 17-20 Aug. Here's my proposed timeline. This will > obviously affect development work some, and since the original > timeline called for us having already released 9.0 buy then ;) > 1. Tuesday evening, around 19:00 central european time, which is 17:00 > GMT or 12:00 EST, I will freeze the current cvs repository. I will do > this by disabling committer login on that box, so please note that > this will also make it impossible for committers to do a "cvs update" > from the authenticated repository. So, per discussion, I'd like to suggest that we have a "quiet time" for say three hours before the repository is closed off, to give interested committers a chance to capture final snapshots of the current repository. IOW, please no more CVS commits after 14:00 GMT tomorrow, Tuesday 8/17. regards, tom lane
Tom Lane wrote: > Magnus Hagander <magnus@hagander.net> writes: > > According to the decision at the developer meeting, the migration to > > git should happen 17-20 Aug. Here's my proposed timeline. This will > > obviously affect development work some, and since the original > > timeline called for us having already released 9.0 buy then ;) > > > 1. Tuesday evening, around 19:00 central european time, which is 17:00 > > GMT or 12:00 EST, I will freeze the current cvs repository. I will do > > this by disabling committer login on that box, so please note that > > this will also make it impossible for committers to do a "cvs update" > > from the authenticated repository. > > So, per discussion, I'd like to suggest that we have a "quiet time" for > say three hours before the repository is closed off, to give interested > committers a chance to capture final snapshots of the current > repository. > > IOW, please no more CVS commits after 14:00 GMT tomorrow, Tuesday 8/17. OK, would someone please send an email to hackers at that time as a reminder? I might not be available then. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. +
On Tue, Aug 17, 2010 at 1:59 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote: > Magnus Hagander <magnus@hagander.net> writes: >> According to the decision at the developer meeting, the migration to >> git should happen 17-20 Aug. Here's my proposed timeline. This will >> obviously affect development work some, and since the original >> timeline called for us having already released 9.0 buy then ;) > >> 1. Tuesday evening, around 19:00 central european time, which is 17:00 >> GMT or 12:00 EST, I will freeze the current cvs repository. I will do >> this by disabling committer login on that box, so please note that >> this will also make it impossible for committers to do a "cvs update" >> from the authenticated repository. > > So, per discussion, I'd like to suggest that we have a "quiet time" for > say three hours before the repository is closed off, to give interested > committers a chance to capture final snapshots of the current > repository. > > IOW, please no more CVS commits after 14:00 GMT tomorrow, Tuesday 8/17. Works for me. I'll send out status reports as I change things, so be sure to check the latest on -hackers if something you thought would work doesn't ;) -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/
On Mon, Aug 16, 2010 at 02:50:35PM -0400, Tom Lane wrote: > Better a memory leak than broken ecpg ;-). Nobody except Michael > is terribly comfortable with that code, so we'd all rather wait for > him to review and apply the patch. This patch was small enough that I felt good about it without having a chance to test it. Anyway, I applied Zoltan's memleak patch to HEAD and 8.4. Michael -- Michael Meskes Michael at Fam-Meskes dot De, Michael at Meskes dot (De|Com|Net|Org) Michael at BorussiaFan dot De, Meskes at (Debian|Postgresql) dot Org ICQ 179140304, AIM/Yahoo/Skype michaelmeskes, Jabber meskes@jabber.org VfL Borussia! Força Barça! Go SF 49ers! Use Debian GNU/Linux, PostgreSQL
On Tue, Aug 17, 2010 at 11:46 AM, Michael Meskes <meskes@postgresql.org> wrote: > On Mon, Aug 16, 2010 at 02:50:35PM -0400, Tom Lane wrote: >> Better a memory leak than broken ecpg ;-). Nobody except Michael >> is terribly comfortable with that code, so we'd all rather wait for >> him to review and apply the patch. > > This patch was small enough that I felt good about it without having a > chance > to test it. Anyway, I applied Zoltan's memleak patch to HEAD and 8.4. What about 9.0? -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/
On Tue, Aug 17, 2010 at 11:50:20AM +0200, Magnus Hagander wrote: > What about 9.0? Will come in a few minutes. I didn't have a checked out version of the 9.0 branch handy in my development environment. Regression test is currently running. Michael -- Michael Meskes Michael at Fam-Meskes dot De, Michael at Meskes dot (De|Com|Net|Org) Michael at BorussiaFan dot De, Meskes at (Debian|Postgresql) dot Org ICQ 179140304, AIM/Yahoo/Skype michaelmeskes, Jabber meskes@jabber.org VfL Borussia! Força Barça! Go SF 49ers! Use Debian GNU/Linux, PostgreSQL
Bruce Momjian <bruce@momjian.us> writes: > Tom Lane wrote: >> IOW, please no more CVS commits after 14:00 GMT tomorrow, Tuesday 8/17. > OK, would someone please send an email to hackers at that time as a > reminder? I might not be available then. Here you go. Per yesterday's discussion, access to the master PG CVS repository will be shut off at approximately 17:00 GMT, three hours from now, to begin the conversion to git. So that all interested people can capture final snapshots of the repository for archival purposes, please do not make any further commits into the CVS repository. Development will resume once the git repository is up and validated. regards, tom lane