Re: how can i work with huge files?

30 MB a very large file? Where are you from? 1980?

Anyway, you avoid loading the entire file into memory by reading it one bit at the time. For instance, by reading it line by line (one can set $/ to define ones line ending), or by reading in a certain number of bytes (set $/ to a reference to an integer, or use (sys)read. You may want to (sys)seek to skip parts of the file - but that only works if you know where to go to.

Comment on Re: how can i work with huge files?

Replies are listed 'Best First'.
Re^2: how can i work with huge files? by morfeas (Novice) on Feb 08, 2005 at 14:47 UTC
yes. the smaller is about 30MB. Now imagine some hundrents or thousants of such files needing to be processed by the script we mentioned. Quite '80s e? Further more say how hours should take, if for ONE 30MB needs 30 mins...you might reffer to 1980 hours... I need something more (P)ractical(erl) Anyway, thinking that tie:File have some potential on it ...	[reply]
Re^3: how can i work with huge files? by Anonymous Monk on Feb 08, 2005 at 15:18 UTC
Since we don't know what you do to a file, how can we asses the "30 minutes to process 30Mb"? Perhaps you have been very clever to do the task at hand in just 30 minutes. Perhaps you have used an extremely slow algorithm, and it could be done in a few seconds. Who knows? Well, you know, but we don't. But I remain, as long as you think that 30 Mb is a "very large file", you're not from this era.	[reply]
A reply falls below the community's threshold of quality. You may see it by logging in.