I'm working on moving all of my CPAN module source code over to github, and I discovered much to my dismay that I didn't have any good code repositories - SVN, CVS, or even RCS. D'oh.

So i decided the next best thing was to rebuild the repositories using CPAN and BACKPAN to get tarfiles of all the releases of each module, then create git repositories treating each tarfile as if it were a commit. I've done "release early, release often" for most of my modules, so I actually had small enough deltas that this was a reasonable thing to do.

After doing a couple by hand, I realized that I could make things much easier for myself by letting Perl grab the names of all the files, unpack them one by one, and add the changed files to the repository each time. I used rsync to make this faster and easier; since it already knows how to crawl down trees and move only the new stuff, it seemed much more reasonable to do it that way.

#!/opt/local/bin/perl use strict; use warnings; my $workdir = shift || die "No work directory supplied\n"; chdir $workdir; my @files = glob('./*.tar.gz'); my %dists; for my $file (@files) { my($dist_name,$release) = ($file =~ m{\A(.*)-(\d+\.\d+)\.tar.gz}) or next; $dist_name =~ s{./}{}; push @{$dists{$dist_name}}, $release; } for my $dist (sort keys %dists) { my $lowered = lc($dist); mkdir $lowered; chdir $lowered; system qq(git init); chdir '..'; for my $release (sort @{ $dists{$dist} } ) { system qq(tar zxvf $dist-$release.tar.gz); system qq(rsync -avv $dist-$release/ $lowered); chdir $lowered; system qq(mv .cvsignore .gitignore); system qq(git diff); system qq(git add .); system qq(git commit -a -m'$release release'); chdir '..'; } }
It's still necessary to create the repos on github and push them up, but the dogwork of cd'ing around, copying files, and committing them to the local repositories becomes trivial.

I should probably add a WWW::Mechanize section at the bottom to create the repos on Github and then push them, but I haven't double-checked the TOS to be sure that would be OK.

Replies are listed 'Best First'.
Re: CPAN dists to git repositories
by BrowserUk (Patriarch) on Mar 13, 2009 at 02:11 UTC

    That's cool.

    One thing though. Wouldn't that run a lot faster if you avoided shelling for every command.

    Something like (bearing in mind I know nothing about what syntax *nix shells accept):

    for my $dist (sort keys %dists) { my $lowered = lc($dist); mkdir $lowered; chdir $lowered; system qq(git init); chdir '..'; for my $release (sort @{ $dists{$dist} } ) { open my $SH, "| sh" or die $!; print $SH <<EOC; tar zxvf $dist-$release.tar.gz rsync -avv $dist-$release/ $lowered chdir $lowered mv .cvsignore .gitignore git diff git add . git commit -a -m'$release release EOC } }

    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.
Re: CPAN dists to git repositories
by MidLifeXis (Monsignor) on Mar 13, 2009 at 14:05 UTC
      Very nice. I'll look into that.
Re: CPAN dists to git repositories
by Daenyth (Novice) on Mar 19, 2009 at 20:35 UTC
    You can use the github command line tool to create the repo automatically, or recode the function into a shell script for yourself.