Yesterday I was feeling ill and bored and decided that it would be fun to watch CPAN get mirrored to my harddrive. I have been wanting to do some statistical analysis on the full CPAN set and I have plenty of room so I figured it would be a suitable distraction for my foggy head. I did a little googling and of course found merlyns useful article on making a Minimal CPAN mirror of the most recent files in the collection.

So of course I read the article, grabbed the code, plugged in my local paths and let it rip. The steady stream of messages about files being downloaded combined with my flu slowly lulled me into a pleasant stupor. Soon I fell sleepy and went for a nap.

Hours later I returned, somewhat refreshed and happy to discover that I was on 'X' and that soon it would be done. Preparations made, the decks cleared for war1, rations for the troops was provided2 and I settled into place to review the downloads, and....

My jaw fell suddenly, while I had been dallying an evil magician had been busy deleting the files, my return was to a log file showing all the precious gems that had been in my hands for the briefest of moments only to be spirited away! Crestfallen I retired from battlefield to analyze what went wrong.....

The responsible code was in two short lines: 

=69=     my $local_file = catfile($LOCAL, split "/", $path);

[download]
and 
=105=    return unless -f and not $mirrored{$File::Find::name};

[download]

for you see, File::Spec::Functions::catfile returns a localized path, and $File::Find::name contains a non localized (in fact not even canonical) string. The solution, albeit probably not the cleanest was to use the following as a replacement for line 105. 

      my ($v,$p,$n)=splitpath($File::Find::name);
      my $name=catfile($v,splitdir($p),$n);
      return unless -f and not $mirrored{$name};

[download]

It seems everyone get bitten by this kind of thing at some point or another...

  1. Empty coffee cups removed ashtry emptied etc.
  2. Food and coffee was provided
---
$world=~s/war/peace/g

Replies are listed 'Best First'.
Re: OS issues get everybody at some point
by PodMaster (Abbot) on May 24, 2005 at 11:47 UTC
    What is a localized path?

    You 2nd snippet is just a really really really really round about way to do 

    my $name = File::Spec->canonpath( $File::Find::name );

    [download]
    See Path::Class for a different interface to File::Spec.

    MJD says "you can't just make shit up and expect the computer to know what you mean, retardo!"
    I run a Win32 PPM repository for perl 5.6.x and 5.8.x -- I take requests (README).
    ** The third rule of perl club is a statement of fact: pod is sexy.

[reply]
[d/l]

      Localized, as in it uses uses the local file sperator. And yeah you're correct about canonpath. I seem to recall that at somepoint the two approaches werent the same, but maybe im confusing it with something else. I was kind of flu-stupid yesterday anyway....

      ---
      $world=~s/war/peace/g

[reply]
Re: OS issues get everybody at some point
by merlyn (Sage) on May 24, 2005 at 11:30 UTC
    Not quite right. The problem (which I've known about since shortly after the article was published) is that File::Find and File::Spec disagree about the which of the two path separarators (backslash or slash) to use in WIndows.

    Thus, the bug is either in File::Spec, or File::Find, or perlport, but I did code the program to specification according to what I could read. Since I've never owned a machine that can run Windows (ever!) (except under virtual PC, ok), there's no way to test it. And the point is that I shouldn't need to test it if I follow all the rules in all those documents.

    -- Randal L. Schwartz, Perl hacker
    Be sure to read my standard disclaimer if this is a reply.

[reply]
      The bug is in File::Find, as it doesn't use File::Spec to concatenate paths, so all paths returned by File::Find need to be run through File::Spec's canonpath

      MJD says "you can't just make shit up and expect the computer to know what you mean, retardo!"
      I run a Win32 PPM repository for perl 5.6.x and 5.8.x -- I take requests (README).
      ** The third rule of perl club is a statement of fact: pod is sexy.

[reply]

        It is a bug to assume that File::Find returns "canonicalized" paths. It doesn't claim that it does. The bug was merlyn's assumption, not a failing in File::Find.

        You could claim that File::Find should be enhanced to conform to File::Spec's concept of canonical path formats. Or you could claim that File::Find should be using File::Spec instead of a ton of hard-coded '/'s. I'd be more likely to agree with the latter than the former, though the end result might be same, at least in some cases.

        - tye        

[reply]
[d/l]
      Since I've never owned a machine that can run Windows

      That _can_ (not _does_)... Wait, you're saying you've never owned a PC of the 8086 chip family? Wow, I didn't know there was anyone left in the first world who hadn't at one time or anther owned a PC. Even most dedicated Macheads I know have a marginally-useful used PC sitting around that they picked up at a flea market or garage sale for five bucks at one time or another in a moment of weakness.

      So, umm. Congratulations.
[reply]
        Howdy!

        Well, being very careful how I say this: I've never owned a PC that ran on an 8086 family chip. I've had custody of such, in my capacity as an officer in a non-profit, and my son (the black sheep, clearly) has a Windoze PC.

        The first PC I owned was an Atari 800. The next was a Fat Mac. That was followed by a Mac IIfx, a StarMax (703e Mac clone), and a 350Mhz G4 Sawtooth AGP box (still in service). Most recently, I splurged on a 15 PowerBook fully loaded.

        ...and my wife and my other son have G4 boxen of their own, and my daughter has a 703e Mac and the StarMax.

        ...so, meet another such...

        yours,
        Michael
[reply]
        Correct. I have never owned anything in the X86 family. Ever. Similar to the other thread, I've owned an Atari 800, then 1200XL, and a SOL-20. Then, I merely used my employer's computers until I could afford my first laptop, a Powerbook 140 (on which I wrote Learning Perl, first edition). I've gone through about 20 Mac laptops since then, and a couple of desktop machines.

        The computer that runs http://www.stonehenge.com is rented, and does in fact run with an X86 chip (an Athalon running OpenBSD). So, I'm careful to say "owned" for my claim.

        -- Randal L. Schwartz, Perl hacker
        Be sure to read my standard disclaimer if this is a reply.

[reply]

Back to Meditations