comment on

Here's a start point:

use warnings;
use strict;

my $chunk;
my $chunkSize = 40;
my $matchText = 'I';
my $partCount = 1;
my $fileSize = 0;

while (my $newChunkSize = read (DATA, $chunk, $chunkSize)) {
    next if $fileSize + $newChunkSize < $chunkSize; # not a complete c
+hunk
    
    # Have at least $chunkSize - look for a stop point
    my $breakPoint = index $chunk, $matchText;
    if ($breakPoint == -1) {
        $fileSize += $newChunkSize;
        next; # Can't change file yet
    }
    
    # Found a stop point, all change
    print substr $chunk, 0, $breakPoint, '';
    
    # close current output file here and create the new file
    
    ++$partCount;
    print "\n\n***Start of new file for part $partCount\n";
    $fileSize = $newChunkSize - $breakPoint;
} continue {
    print $chunk;
}

__DATA__
[download]

Hello!

Let me say first I've visisted this site for a long time: you guys are
+ a great
font of knowledge the the n00b programmer like myself.

And with that...

I am working a program at my internship for processing some huge text 
+files(
1.5 gig ish). The goal is to split the file into 200mb chunks for use 
+in other
programs.

Searching google we found some code to use that works well for splitti
+ng files
( "http://search.cpan.org/src/CWEST/ppt-0.14/bin/split" )

My question comes at the part...

while (read (INFILE, $chunk, $count) == $count) {
    $fh = nextfile ($prefix);
    print $fh $chunk;
    }
[download]

I want to edit this so that, when it hits the chunk size, it will keep
+ going
UNTIL it hits a certain string. Where it will then resume creating ano
+ther chunk.

My confusion comes with how that while loop flag condition works. My e
+xperience
with Perl ended before ever seeing something like this.

I would be grateful to be pointed to some resource or something to loo
+k up to
set me in the right direction. As, well, I really would like to know w
+hat the
heck I'm doing...heh...

Thanks,

Mertz
[download]

Prints:

Hello!

Let me say first 

***Start of new file for part 2
I've visisted this site for a long time: you guys are a great
font of knowledge the the n00b programmer like myself.

And with that...



***Start of new file for part 3
I am working a program at my internship for processing some huge text 
+files(
1.5 gig ish). The goal is to split the file into 200mb chunks for use 
+in other
programs.

Searching google we found some code to use that works well for splitti
+ng files
( "http://search.cpan.org/src/CWEST/ppt-0.14/bin/split" )

My question comes at the part...

while (read (

***Start of new file for part 4
INFILE, $chunk, $count) == $count) {
    $fh = nextfile ($prefix);
    print $fh $chunk;
    }
[download]



***Start of new file for part 5
I want to edit this so that, when it hits the chunk size, it will keep
+ going
UNT

***Start of new file for part 6
IL it hits a certain string. Where it will then resume creating anothe
+r chunk.

My confusion comes with how that while loop flag condition works. My e
+xperience
with Perl ended before ever seeing something like this.



***Start of new file for part 7
I would be grateful to be pointed to some resource or something to loo
+k up to
set me in the right direction. As, well, 

***Start of new file for part 8
I really would like to know what the
heck 

***Start of new file for part 9
I'm doing...heh...

Thanks,

Mertz
[download]

DWIM is Perl's answer to Gödel

In reply to Re: File Processing... by GrandFather
in thread File Processing... by Anonymous Monk

Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!

Titles consisting of a single word are discouraged, and in most cases are disallowed outright.

Read Where should I post X? if you're not absolutely sure you're posting in the right place.

Please read these before you post! —

Posts may use any of the Perl Monks Approved HTML tags:

a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr

You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)

	For:		Use:
	&		`&`
	<		`<`
	>		`>`
	[		`[`
	]		`]`

Link using PerlMonks shortcuts! What shortcuts can I use for linking?

See Writeup Formatting Tips and other pages linked from there for more info.