Обсуждение: having problem in rsync'ing cvs

Hi all,<br /><br />    I was rsyncing a fresh copy of CVS repository, and suddenly, midway (around heap.c) rsync
complained.When after many retries I couldn't start it, I restarted the OS.<br /><br />    Since it is a VM running
insideVista (from where I could ping and browse <a href="http://anoncvs.postgresql.org">anoncvs.postgresql.org</a>), I
restartedthe Host OS too.<br /><br />Here's the configuration:<br />Host OS: Windows Vista<br />VM Software: Innotek
VirtualBox<br/>Guest OS: Ubuntu 7.10<br /><br />And here's the error I am getting:<br /><br
/>gurjeet@g-u-v:~/dev/repositories$rsync -avzCH --delete anoncvs.postgresql.org::pgsql-cvs pg_cvsroot/<br /> rsync:
getaddrinfo:<a href="http://anoncvs.postgresql.org">anoncvs.postgresql.org</a> 873: Name or service not known<br
/>rsyncerror: error in socket IO (code 10) at clientserver.c(104) [receiver=2.6.9]<br
/>gurjeet@g-u-v:~/dev/repositories$<br /><br />Is the rsync daemon on anoncvs down? Is everyone else able to do
rsync?<br/><br />Thanks in advance,<br /><br />Best regards,<br />-- <br />gurjeet[.singh]@EnterpriseDB.com<br
/>singh.gurjeet@{gmail | hotmail | indiatimes | yahoo }.com<br /><br />EnterpriseDB <a
href="http://www.enterprisedb.com">http://www.enterprisedb.com</a><br/><br />Mail sent from my BlackLaptop device  

On 27/03/2008, Gurjeet Singh <singh.gurjeet@gmail.com> wrote:
> Is the rsync daemon on anoncvs down? Is everyone else able to do rsync?
>

Possibly related; the Postgres git repository at
http://repo.or.cz/w/PostgreSQL.git is showing the last commit at 25
hours ago.  It's usually a bit more spry than that.

Gurjeet Singh escribió:

> Is the rsync daemon on anoncvs down? Is everyone else able to do rsync?

I can rsync with no issue.

-- 
Alvaro Herrera                                http://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

* Brendan Jurd <direvus@gmail.com> [080326 10:19]:
> On 27/03/2008, Gurjeet Singh <singh.gurjeet@gmail.com> wrote:
> > Is the rsync daemon on anoncvs down? Is everyone else able to do rsync?
> >
> 
> Possibly related; the Postgres git repository at
> http://repo.or.cz/w/PostgreSQL.git is showing the last commit at 25
> hours ago.  It's usually a bit more spry than that.

Oops - no, that's just me...

I was recently made aware of
this:http://repo.or.cz/w/PostgreSQL.git?a=commit;h=69db64c737012a8d2d6fbcce3ace7136cb2bc85f

I started digging around to figure this out on Tuesday.

It appears as if the "rsync" mirror of CVS is not always "good".  It
seems like "long running" CVS operations (like I'm guessing a full tree
"tag" of REL8_3_STABLE) aren't mirrored "atomically".  Of course, CVS
isn't atomic, so we can't really blame it.

What appears to have happened is that my rsync caught the rsync mirror
when the tree was "not all tagged", so when the fromcvs went about
making the new branch on the first appearance of REL8_3_STABLE, it had
to remove a bunch of files from the branch because they were *not*
tagged with that symbol in CVS (or at least, not presently tagged with
that symbol in the rsync mirror of CVS)...

I would guess that any incremental CVS mirror/conversion is going to hit
this at some random time too.  Of course, the risk of hitting it goes up
as the frequency of your rsync updates go up.  

And I just forgot to re-enable my cron after I finished looking at it.

BTW, anybody following the GIT mirror, the REL8_3_STABLE branch has been
re-wound, you you'll probably have to force update it (git fetch -f) if
you only accept fast forward updates on fetches (the default).

And if you have patches based on REL8_3_STABLE, you'll need to rebase
them too.  Of course, seeing as the tree in REL8_3_STABLE mirror was
broken, I suspect the set of people using it was 0.

a.


-- 
Aidan Van Dyk                                             Create like a god,
aidan@highrise.ca                                       command like a king,
http://www.highrise.ca/                                   work like a slave.

On 28/03/2008, Aidan Van Dyk <aidan@highrise.ca> wrote:
>  And I just forgot to re-enable my cron after I finished looking at it.
>

Ah, the old "post-maintenance-disabled-cron" gaff.  One of my personal
favourites. =)

I'm not sure that the git repos has fully recovered.  There seems to a
block of commits missing, between 2008-03-25 13:09 and 2008-03-27
17:24 UTC.

Looking at the CVS logs, there was definitely commit action in that
timeframe, but none of it is showing up on the git shortlog.

Cheers,
BJ

* Brendan Jurd <direvus@gmail.com> [080327 16:08]:

> I'm not sure that the git repos has fully recovered.  There seems to a
> block of commits missing, between 2008-03-25 13:09 and 2008-03-27
> 17:24 UTC.
> 
> Looking at the CVS logs, there was definitely commit action in that
> timeframe, but none of it is showing up on the git shortlog.

OK, I'll take another look at it tonight.

a.

-- 
Aidan Van Dyk                                             Create like a god,
aidan@highrise.ca                                       command like a king,
http://www.highrise.ca/                                   work like a slave.

>  BTW, anybody following the GIT mirror, the REL8_3_STABLE branch has been
>  re-wound, you you'll probably have to force update it (git fetch -f) if
>  you only accept fast forward updates on fetches (the default).

Thanks! now i can ditch the git.or.cz mirror

>  And if you have patches based on REL8_3_STABLE, you'll need to rebase
>  them too.  Of course, seeing as the tree in REL8_3_STABLE mirror was
>  broken, I suspect the set of people using it was 0.
>
>  a.

On Thu, Mar 27, 2008 at 5:34 PM, Alex Hunsaker <badalex@gmail.com> wrote:
> >  BTW, anybody following the GIT mirror, the REL8_3_STABLE branch has been
>  >  re-wound, you you'll probably have to force update it (git fetch -f) if
>  >  you only accept fast forward updates on fetches (the default).
>
>  Thanks! now i can ditch the git.or.cz mirror
>

Err oops I was confused, i was talking about git.postgresql.org where
REL8_3_STABLE is about 6 weeks old

* Alex Hunsaker <badalex@gmail.com> [080327 19:38]:
> On Thu, Mar 27, 2008 at 5:34 PM, Alex Hunsaker <badalex@gmail.com> wrote:
> > >  BTW, anybody following the GIT mirror, the REL8_3_STABLE branch has been
> >  >  re-wound, you you'll probably have to force update it (git fetch -f) if
> >  >  you only accept fast forward updates on fetches (the default).
> >
> >  Thanks! now i can ditch the git.or.cz mirror
> >
> 
> Err oops I was confused, i was talking about git.postgresql.org where
> REL8_3_STABLE is about 6 weeks old

Ya, Peter E asked me to help look at that for a while ago too.  I couldn't
figure out why it's not updating either.  

-- 
Aidan Van Dyk                                             Create like a god,
aidan@highrise.ca                                       command like a king,
http://www.highrise.ca/                                   work like a slave.

* Brendan Jurd <direvus@gmail.com> [080327 16:36]:
> Ah, the old "post-maintenance-disabled-cron" gaff.  One of my personal
> favourites. =)
> 
> I'm not sure that the git repos has fully recovered.  There seems to a
> block of commits missing, between 2008-03-25 13:09 and 2008-03-27
> 17:24 UTC.
> 
> Looking at the CVS logs, there was definitely commit action in that
> timeframe, but none of it is showing up on the git shortlog.

OK, so it should all be valid again.  

Sorry for any issues I caused to those trying to use that mirror
regularly...

a.
-- 
Aidan Van Dyk                                             Create like a god,
aidan@highrise.ca                                       command like a king,
http://www.highrise.ca/                                   work like a slave.

On 29/03/2008, Aidan Van Dyk <aidan@highrise.ca> wrote:
> * Brendan Jurd <direvus@gmail.com> [080327 16:36]:
>  >
>  > Looking at the CVS logs, there was definitely commit action in that
>  > timeframe, but none of it is showing up on the git shortlog.
>
> OK, so it should all be valid again.
>

Looks good to me.  Thanks Aidan.

BJ

* Brendan Jurd <direvus@gmail.com> [080328 11:16]:
> On 29/03/2008, Aidan Van Dyk <aidan@highrise.ca> wrote:
> > * Brendan Jurd <direvus@gmail.com> [080327 16:36]:
> >  >
> >  > Looking at the CVS logs, there was definitely commit action in that
> >  > timeframe, but none of it is showing up on the git shortlog.
> >
> > OK, so it should all be valid again.
> >
> 
> Looks good to me.  Thanks Aidan.

And, of course, after "fixing" the missing one, I forgot to enable the
cronjob *again*...

It's running now again.

Really.

a.

-- 
Aidan Van Dyk                                             Create like a god,
aidan@highrise.ca                                       command like a king,
http://www.highrise.ca/                                   work like a slave.

On Mar 27, 2008, at 10:47 AM, Aidan Van Dyk wrote:
> I was recently made aware of this:
>     http://repo.or.cz/w/PostgreSQL.git? 
> a=commit;h=69db64c737012a8d2d6fbcce3ace7136cb2bc85f
>
> I started digging around to figure this out on Tuesday.
>
> It appears as if the "rsync" mirror of CVS is not always "good".  It
> seems like "long running" CVS operations (like I'm guessing a full  
> tree
> "tag" of REL8_3_STABLE) aren't mirrored "atomically".  Of course, CVS
> isn't atomic, so we can't really blame it.
>
> What appears to have happened is that my rsync caught the rsync mirror
> when the tree was "not all tagged", so when the fromcvs went about
> making the new branch on the first appearance of REL8_3_STABLE, it had
> to remove a bunch of files from the branch because they were *not*
> tagged with that symbol in CVS (or at least, not presently tagged with
> that symbol in the rsync mirror of CVS)...
>
> I would guess that any incremental CVS mirror/conversion is going  
> to hit
> this at some random time too.  Of course, the risk of hitting it  
> goes up
> as the frequency of your rsync updates go up.


Hrm... is there a way to momentarily lock-out access to CVS? What I'm  
thinking is to have a script that periodically locks CVS access,  
takes a snapshot of the tree (preferably via a filesystem snapshot),  
and then unlocks. That snapshot would then be used to drive the  
mirrors. That would ensure that mirrors always had an atomic view of  
things.
-- 
Decibel!, aka Jim C. Nasby, Database Architect  decibel@decibel.org
Give your computer some brain candy! www.distributed.net Team #1828