Hi I am new to perl.My requirement is to split huge file(3GB) into multiples of 200MB files.I have used the option of perl built-in functions like read()/sysread().The files are getting split to 200MB files and the basic logic is working what I have written the code.But since I am using the read function, the data is getting transferred in bytes and in each file, there is a chunk of information which does not actually terminate at end of line and next line does not start from the beginning of the line.Please help!! I need to know if I use read function, can I transfer the data exactly for the end of line?

The output looks like this 2011/11/30 @ 18:08:52,103 @ -> GetLogicalServerByIdTask-124191 - Impl.addLogEntry 2011/11/30 @ 18:08:52,112 @ -> [ActiveMQ Session Task 2011/11/30 @ 18:12:12,042 @ -> ActiveMQ Session Task - WARN le for synchronizing its resources. 


my $filesize_in_MB=0;
    my $file_size_compare=100;
    my $filename;
    my $filesize;
    my $block_size=131072;
    my $file_size_sorted= -s $file_sorted;
    my $file_size_sorted_MB=$file_size_sorted/(1024*1024);
my $buffer;
my $count=15;
open FH_sort, "$file_sorted";


for (my $i=1;$i <= $count;$i++)
{


while($filesize_in_MB <= $file_size_compare)
{
my $rv=read(FH_sort,$buffer,$block_size); #or die "$?";
#my $rv=read(FH_sort,$buffer,$block_size,O_APPEND ); #or die "$?";
#print $rv;
if (!eof(FH_sort) && ($rv <= $block_size))
{
open FH_split, ">>sort_split$i" or die "$!";

print FH_split $buffer;
         $filename="sort_split$i";
         $filesize= -s $filename;
         $filesize_in_MB=$filesize/(1024*1024);
close FH_split;
}
else
{
open FH_split, ">>sort_split$i" or die "$!";
print FH_split $buffer;
close FH_split;

last;
}
}
 $filesize_in_MB=0;

}
close FH_sort;

[download]

In reply to Large file split into smaller using sysread() by rkshyam

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.