comment on

Hi monks, I need to parse and process a very big file. What i need is use buffered read for speed (sysread). My problem is retrieving the lines after the read: the buffer stops in the middle of a line -obvious-, but:

#!/usr/bin/perl -w
open my( $fh ), '<', "/usr/local/ffpde/logs/pruebas3.log";
my $buffer;
while (sysread $fh, $buffer, 100) {
        my @lines = split(/"\n"/, $buffer);
        print @lines;
        sleep 1;
}
[download]

#!/usr/bin/perl -w
open my( $fh ), '<', "/usr/local/ffpde/logs/pruebas3.log";
my $buffer;
while (sysread $fh, $buffer, 100) {
        my @lines = split(/"\n"/, $buffer);

        for $l (@lines) {
                print "$l\n";
                sleep 1;
        }

}
[download]

Jul 26 10:45:25 - Sergio, 33 | Informático
Jul 26 11:45:25 - Angel, 23 | Encofrador
Jul 26 12:45:25
- Sergio, 52 | Repartidor
Jul 26 12:55:25 - Sergio, 18 | Repartidor
Jul 26 13:25:25 - Angel, 42 | P
anadero
Jul 26 13:35:25 - Dario, 34 | Informático
Jul 26 15:45:25 - Luis, 26 | Repartidor
Jul 26 16
:25:25 - Mabel, 41 | Azafata
Jul 26 17:29:25 - Laura, 19 | Investigadora
Jul 26 10:45:25 - Sergio, 3
3 | Informático
Jul 26 11:45:25 - Angel, 23 | Encofrador
Jul 26 12:45:25 - Sergio, 52 | Repartidor
Jul 26 12:55:25 - Sergio, 18 | Repartidor
Jul 26 13:25:25 - Angel, 42 | Panadero
Jul 26 13:35:25 - D
ario, 34 | Informático

It splits the line where the buffer stopped. (oops)

The buffer size doesn't matter, just for the example to see 2-3 lines of read, its the same with 4096 bytes.

So: is there any method to avoid this cut-line fact when reading with buffers? Which is the best method to load a array with the line-by-line correct content of the file. I must use buffers for speed, simple open lacks of speed for me.

Thanks in advance monks!

In reply to Line by line buffered read by muyprofesional

Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!

Titles consisting of a single word are discouraged, and in most cases are disallowed outright.

Read Where should I post X? if you're not absolutely sure you're posting in the right place.

Please read these before you post! —

Posts may use any of the Perl Monks Approved HTML tags:

a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr

You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)

	For:		Use:
	&		`&`
	<		`<`
	>		`>`
	[		`[`
	]		`]`

Link using PerlMonks shortcuts! What shortcuts can I use for linking?

See Writeup Formatting Tips and other pages linked from there for more info.