Re: Alternate for "open"

Please show the code you are using to open and read your file.

open() shouldn't use up your memory, but reading the file might. Perhaps you are trying to read it all into memory at once? That will cause problems with large files; better to just handle one line at a time in memory.

Try something like:

my $filepath = '/path/to/huge/file';
open my $fh, '<:encoding(UTF-8)', $filepath or die $!;
while ( my $line = <$fh> ) {
    # process one line at a time
}
close $fh or die $!;
[download]

HTH!

The way forward always starts with a minimal test.

Comment on Re: Alternate for "open" Download Code

Replies are listed 'Best First'.
Re^2: Alternate for "open" by ravi45722 (Pilgrim) on Nov 16, 2015 at 08:16 UTC
`foreach $file (@cdr_list) { chomp $file; open (FP,"$file") or die "Could not open $file\n"; $first=1; while ($line=<FP>){ chomp $line;` [download] Here is the code I am using to read the file. Actually I am running this program in server which contains 64GB of RAM. Whenever the program started running the memory usage jumps to 98% around from 25-30%. I think it's because of the files. I correct or I have to concentrate on other parts???	[reply] [d/l]
Re^3: Alternate for "open" by Corion (Patriarch) on Nov 16, 2015 at 08:27 UTC
You could try to confirm your assumption by running the above minimal code, only reading the file and doing nothing else.	[reply]
Re^3: Alternate for "open" by 1nickt (Canon) on Nov 16, 2015 at 15:18 UTC
Well, with the information you've added (the RAM on your box, the size of the files) it's clear that opening a file is not going to use up all your memory. Did you try Corion's suggestion and remove all code except the loop through the files and the `open()` and `close()` statements? From here it looks a lot like it's the rest of your code (that you didn't share) that is causing the problem. Once you've verified for yourself that it's not the `open()` call, you'll have to share the actual code you're using if you would like to get help. The way forward always starts with a minimal test.	[reply] [d/l] [select]
Re^4: Alternate for "open" by ravi45722 (Pilgrim) on Nov 17, 2015 at 04:53 UTC
Its not possible to share the total code. Its nearly 2000 lines with 26 config files.(Total Config files size is 2 MB). Tell me a best way how to show all my code. I will try that	[reply]
Re^5: Alternate for "open" by Corion (Patriarch) on Nov 17, 2015 at 07:35 UTC
Re^3: Alternate for "open" by Preceptor (Deacon) on Nov 16, 2015 at 17:24 UTC
Your lack of `use strict; use warnings;` is likely hiding a problem from you	[reply] [d/l]