in reply to -s takes too long on 15,000 files

The answer to this turns out to be some support for some rarely required functionality meant that perls stat had to open and then close each one of those files. Later perls will/should support ${^WIN32_SLOPPY_STAT}=1; which will disable this behaviour.

Sorry about how long it took to figure it out. :-)

---
$world=~s/war/peace/g

Replies are listed 'Best First'.
Re^2: -s takes too long on 15,000 files
by BrowserUk (Patriarch) on Feb 27, 2006 at 17:27 UTC
    some support for some rarely required functionality

    Care to explain what the functionality was/is?

    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    Lingua non convalesco, consenesco et abolesco. -- Rule 1 has a caveat! -- Who broke the cabal?
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.
[reply]

      The POSIX system on NT/Win2k allows the creation of hardlinks to files if the user doing the creation has Backup/Restore privs. Jan Dubois put a patch into win32 perl that allows perlfunc:link to create symbolic links using this feature. (That this is possible is poorly and confusingly documented, both by Microsoft and by ActiveState/Perl. For the latter you can find it documented in perlport)

      One drawback of this however is that various attributes somehow dont propagate to the linked file unless its opened first (a caching bug apparently). So in order to cater for the 0.00000000000001 percent of the files out there that are hardlinks on Win32, perl was made to open and close every file it stats on that OS. :-(

      Anyway, 5.10 will support the new var (which I had the honor of naming) which will disable this behaviour. And so hopefully will the next 5.8.x release. Along with using the newish sitecustomize.pl support you can set this variable to true for your perl and bypass this behaviour if you know it wont be a problem.

      This should mean some signifigant speedups. According to Jan (who quite naturally beat me to an implementation for this) the new behaviour shows

      The STRICT test uses 30-50% more wall clock time on a local NTFS system, 100% more against a Samba server and about 200% more against a Windows file server. This is very approximate, as times even on the local disk vary widely.

      There are also JUNCTIONS on later NT/Win32 versions, but i dont think they cause stat a problem.

      ---
      $world=~s/war/peace/g

[reply]
        One drawback of this however is that various attributes somehow dont propagate to the linked file unless its opened first (a caching bug apparently).

        Thanks for the update.

        I found the MS page that relates to this. The way they describe it, it reads more as a limitation than a bug per se, but it is obscure enough that your bypass mode might make the better default--at least on native builds. I would guess that it may see more use on cygwin builds?

        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        Lingua non convalesco, consenesco et abolesco. -- Rule 1 has a caveat! -- Who broke the cabal?
        "Science is about questioning the status quo. Questioning authority".
        In the absence of evidence, opinion is indistinguishable from prejudice.
[reply]

In Section Seekers of Perl Wisdom