I need to parse a string which is larger than 2GB. Unfortunately "split()" fails with a "Split loop" error.

Looking for an alternate. Here is the minimized test case. due to the nature of the original code, the "read()" of the file cannot be changed.

thanks in advance for any help...

EDIT: 6/30/2013:
To clarify, this is trimmed down from the original to illustrate the problem with split.
I can only modify "BigParse". I have no control of the strings passed in.
The real code processes each split until end of data. The print statement in the loop is just to trim down the code.

Perl version: 5.12.4

Data File is binary file with embedded newline characters "\n".

$ wc -c data
 2753110808 data
$ wc -l data
 2753111 data
[download]

#!/usr/bin/perl

$FILE = "data";
open (INFILE, "$FILE") || die "Not able to open the file: $FILE \n";
binmode INFILE;

my $map;
read(INFILE, $map, 2147483648);
# Using this read instead, everything works.
# read(INFILE, $map, 2147483630);
BigParse($map);
exit;

sub BigParse {
   my $map = shift;
   print "string length = ", length($map), "\n";
# This fails with "Split loop" error message.
   foreach my $l (split("\n", $map)) {
     print $l;
   }
   return;
}
[download]

---

---

• Are you posting in the right place? Check out Where do I post X? to know for sure.
• Posts may use any of the Perl Monks Approved HTML tags. Currently these include the following:

  <code> <a> <b> <big> <blockquote> <br /> <dd> <dl> <dt> <em> <font> <h1> <h2> <h3> <h4> <h5> <h6> <hr /> <i> <li> <nbsp> <ol> <p> <small> <strike> <strong> <sub> <sup> <table> <td> <th> <tr> <tt> <u> <ul>

• Snippets of code should be wrapped in <code> tags not <pre> tags. In fact, <pre> tags should generally be avoided. If they must be used, extreme care should be taken to ensure that their contents do not have long lines (<70 chars), in order to prevent horizontal scrolling (and possible janitor intervention).
• Want more info? How to link or How to display code and escape characters are good places to start.