Re: Can I split a 10GB file into 1 GB sizes using my repeating data pattern

As long as you don't try to read the whole file into memory at once, you will be fine.

Just read a line at a time and open a new outputfile whenever you see that the size is too big and the current line starts with "100". Something like this:
~~while (defined $line = <$inputFileHandle>)~~

while ($line = <$inputFileHandle>)
{
  if ($sizeSoFar > 1e9 and $line =~ /^100/)
  {
    $outputFileName++;
    open $outputFileHandle, '>', $outputFileName or die "Cannot open $
+outputFilename for writing: $!\n";
    $sizeSoFar = 0;
  }
  print $outputFilehandle $line;
  $sizeSoFar += length($line);
}
[download]

Comment on Re: Can I split a 10GB file into 1 GB sizes using my repeating data pattern Download Code

Replies are listed 'Best First'.
Re^2: Can I split a 10GB file into 1 GB sizes using my repeating data pattern by jwkrahn (Abbot) on Jul 22, 2009 at 19:10 UTC
`while (defined $line = <$inputFileHandle>)` [download] Because the `=` operator has higher precedence than the defined operator you need to either enclose the assignment in parentheses: `while (defined( $line = <$inputFileHandle> ))` Or because perl does the defined test by default then just omit it and perl will do the right thing: `while ($line = <$inputFileHandle>)`	[reply] [d/l] [select]