in reply to Dealing with huge text string

Read it one record at a time:

open(INPUT,"filename.txt") or die "Will not open input: $!"; open(OUTPUT,">output.txt") or die "Will not open output: $!"; local $/ = \164; while( <INPUT> ) { ## Updated per jwkrahn's post below print OUTPUT "$_\n"; } close OUTPUT; close INPUT;

BTW: The semicolon after open(OUTPUT,">output.txt"); is kinda a givaway that you didn't have the error checking :)

As a one-liner:

perl -ple"BEGIN{$/=\164}" filename.txt >output.txt

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.
"Too many [] have been sedated by an oppressive environment of political correctness and risk aversion."

Replies are listed 'Best First'.
Re^2: Dealing with huge text string
by jwkrahn (Abbot) on Mar 28, 2008 at 13:32 UTC

    Change  while( <IN> ) { to  while( <INPUT> ) { and it may work better.

Re^2: Dealing with huge text string
by PhilFromIndy (Initiate) on Mar 28, 2008 at 14:03 UTC
    Took a few seconds to run, thanks!
    The superfluous semicolon ended up in the opening of the output file while I was changing the names of the files to protect the innocent.
Re^2: Dealing with huge text string
by mobiusinversion (Beadle) on Mar 28, 2008 at 19:28 UTC
    That will certainly fail in many circumstances. See Ikegami's post and my example. If you are unlucky enough to break a wide character in half with 164 byte reads, well, that would suck