Re: How to improve speed of reading big files

Hardware?
What is the data stored on? A local hard drive? Somewhere on the network?
If it's a local hard drive, loo up the model and make. Look up the specs. What speed does it read/write at? Maybe upgrade that hard drive to something better.
What about the processor? What speed is it? Is it 32 or 64 bit.. etc..
Profiling?
How about while it runs your check # top
How about profiling the script?
Changing the code around a bit?
What if you first read he entire contents of the file to an array- in one step. At that point the data will be in memory. And then process the stuff.. It may have an impact on the speed. Who knows. It's worth a try. ( Illuminatus suggested this )

My semi educated guess is that anything short of a hardware change won't shave off more than 20 secs.

Comment on Re: How to improve speed of reading big files

Replies are listed 'Best First'.
Re^2: How to improve speed of reading big files by korlaz (Novice) on Sep 18, 2009 at 08:01 UTC
Hardware: local drive SCSI (can't change it) Profiling: writing in parenthesis Changing the code around a bit: the file is already read as an array (grep function on the handle): `grep { /Running\|Dump\|FromCB\|Update/o } <$fh>` [download]	[reply] [d/l]
Re^3: How to improve speed of reading big files by leocharre (Priest) on Sep 18, 2009 at 15:42 UTC
Are you suggesting the items in your original post (secs), is your profiling output? Are you sure you read Devel::Dprof? Do you understand that you can change your code around to get real nice detail about what's taking time, etc?	[reply]