Beefy Boxes and Bandwidth Generously Provided by pair Networks
more useful options
 
PerlMonks  

Re: Re: Faster Method for Gathering Data

by herveus (Parson)
on Jul 31, 2003 at 15:05 UTC ( #279601=note: print w/replies, xml ) Need Help??


in reply to Re: Faster Method for Gathering Data
in thread Faster Method for Gathering Data

Howdy!

Your problem is most likely not really related to perl, it is a filesystem thing, where lookups are made in linear time with regards to the number of files in a directory

I've run into what I think is similar behavior. I have some CDs that have something like 11,000 files on them, all in a single directory. On a Windows or MacOS 9 box, I saw excruciatingly slow access times for files down in the list. The first few hundred were plenty zippy, but the farther I got into the list, the slower the access.

Doing the same access on a Solaris box or MacOSX yielded pleasantly surprising results. File lookups were more like constant time instead of proportional to how far into the list the name was.

I suspect that the problem is exacerbated by using a "slow" medium, like CD-ROM or network volumes.

yours,
Michael

  • Comment on Re: Re: Faster Method for Gathering Data

Replies are listed 'Best First'.
Re: Re: Re: Faster Method for Gathering Data
by Cine (Friar) on Jul 31, 2003 at 15:10 UTC
    Network, yes. CDROM no. The difference is that the meta data on the CDROM can be cached, whereas the network drive has to recheck it.

    T I M T O W T D I

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://279601]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others taking refuge in the Monastery: (2)
As of 2022-10-01 04:41 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    I prefer my indexes to start at:




    Results (126 votes). Check out past polls.

    Notices?