Hello folks. I got a problem that I can't solve. I am working on a pretty cool encryption/decryption system. OK, I am reading a text file which consists of just 0's and 1's...no newlines or whitespace. The file can range in size and get to be VERY large. For this example, the file I am reading is 2,387,250 bytes in size. I need to get every byte of the file, so here are 3 different methods I tried using, and each one eats up ALOT of RAM: 
sub test_loop_1{
    ########   RAM used: 190 MB
    my(@all, $elements);

    @all = ();

    open(FILE, $file) or die;
    while(<FILE>){
       push @all, /\d/og;
    }
    close(FILE);

    $elements = @all;
    print $elements;  # Just for confirmation - prints 2387250
}

sub test_loop_2{
    ########   RAM used: 185 MB
    my(@all, @all2, $all, $elements);
    
    open(FILE, $file) or die;
    @all = <FILE>;
    close(FILE);
    
    $all = join('', @all);
    @all2 = split('', $all);
    $elements = 0;
     
    foreach(@all2){
       $elements ++;
    }

    print $elements;  # Just for confirmation - prints 2387250
}

sub test_loop_3{
    ########    RAM used: 120 MB
    my(@all, @all2, $all, $elements);

    open(FILE, $file) or die;
    @all = <FILE>;
    close(FILE);

    $all = join('', @all);
    @all2 = ();

    for($i = 1; $i <= length($all); $i ++){ 
      push @all2, substr($all, $i, 1);
    }
   
    $elements = @all2;
    print $elements;  # Just for confirmation - prints 2387250
}

[download]
Is there any way around this hogging of RAM, or being that the file is just so large in size, am I gonna have to deal with it?

Thanks for any advice,
David
http://www.trixmaster.com

In reply to Eating RAM problem by TheFifthDeuce

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.