Re: Out of memory problems

How are you reading the file? and/or how many bytes are you reading at a time?

Are you using <FH>, with or without changing $/? or read/sysread(FH)?

If it is possible then I suggest you read only parts of your file at a time (1 kb?, 100 kb?, 1 mb, ...?). You can do this either by using read/sysread or by setting $/ to a reference to an integer: eg: $/ = \1024;, this way <FH> will/should only read 1024 bytes.

Comment on Re: Out of memory problems Download Code

Replies are listed 'Best First'.
Re^2: Out of memory problems by tperdue (Sexton) on Oct 21, 2004 at 10:40 UTC
Here is the code I'm using. Hopefully it's help. #!/usr/local/bin/perl -w undef $/; #Clear the record seperator. $\| = 1; #Don't Buffer data. Write out when available. open IN, "$ARGV[0]"; binmode IN; #Use binary mode so windows/DOS won't complain #Convert original file into binary and write to $final while (<IN>) { #Loop thru file and convert file to 1's/0's $array = unpack("B", $_); open OUT, ">>tmp"; print OUT "$array"; close OUT; undef $array; } close IN; print "Finished unpacking data.\n\n"; undef $/; open IN, "tmp"; #Extract the necessary data bits while (<IN>) { $_ =~ s/11110100.{8}(.{1520})11110100.{8}(.{464}).{1056}/$1$2/g; $final - pack("B", $_); #Conver data back to original binary for +mat open OUT, ">>$ARGV[1]"; print OUT "$final"; close OUT; undef $final; } close IN; unlink "tmp"; print "Finished converting to binary.\n\n"; [download] 20041021 Janitored by Corion: Added code tags	[reply] [d/l]

Replies are listed 'Best First'.

Re^2: Out of memory problems
by tperdue (Sexton) on Oct 21, 2004 at 10:40 UTC

#!/usr/local/bin/perl -w

undef $/; #Clear the record seperator.
$| = 1;  #Don't Buffer data.  Write out when available.

open IN, "$ARGV[0]";

binmode IN;  #Use binary mode so windows/DOS won't complain

#Convert original file into binary and write to $final

while (<IN>) { #Loop thru file and convert file to 1's/0's
     $array = unpack("B*", $_);
     open OUT, ">>tmp";
     print OUT "$array";
     close OUT;
     undef $array;
}

close IN;

print "Finished unpacking data.\n\n";

undef $/;

open IN, "tmp";

#Extract the necessary data bits

while (<IN>) {

     $_ =~ s/11110100.{8}(.{1520})11110100.{8}(.{464}).{1056}/$1$2/g;

     $final - pack("B*", $_); #Conver data back to original binary for
+mat

     open OUT, ">>$ARGV[1]";

     print OUT "$final";

     close OUT;

     undef $final;
}

close IN;

unlink "tmp";

print "Finished converting to binary.\n\n";
[download]

20041021 Janitored by Corion: Added code tags

[reply]
[d/l]