You said: I am working on an encryption system where each ASCII char is assigned a certain number of bits, so for example, if the text to be encrypted is 1000 bytes, then after encryption that text will be converted to 36000 bytes consisting of just 0's and 1's.

Is it the case that the encryption system requires access to the entire data stream in order to work at all? If encrypting, say, 10 sets of 100 bytes (producing 10 sets of 3600 bytes) works as well as cranking a lump of 1000 bytes into 36000, then you should just read, process and output a small portion of data at a time, rather than trying to hold an entire file -- with massive amounts of wasted bits -- in memory at one time.

Apart from that -- I'm sorry but... -- if memory consumption is an issue, and forcing some particular method of bit padding is a requirement, I'd use C rather than Perl.

update: Maybe what you want is sysread, to bring a stated number of bytes into an input scalar variable; e.g.: 

while ( $n_bytes_read = sysread( FILE, $inpbuf, 32 ) > 0 )
{
   if ( $n < 32 ) { # must be the last chunk
      # ... maybe this needs special treatment
   }
   process_input_bytes( $inpbuf );
}

[download]

In reply to Re: Re: Re: Eating RAM problem by graff
in thread Eating RAM problem by TheFifthDeuce

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.