in reply to Re: hang caused by read / readline?
in thread hang caused by read / readline?

The file does contain lines. However, there is a chunk of chr(0) that has been appended (by a process outside my control) to the front of the first line, and I didn't realize that there was an INSANE quantity of chr(0) until I was able to demonstrate it just a bit ago. So, this has exposed the next layer of the problem. I now know why readline() was choking (and why I thought that read() was choking even though it really wasn't, though that's not relevant). So, what's the best way to quickly breeze past the chr(0) mess -- now at 385MB and counting -- to find the start of the real data?

Replies are listed 'Best First'.
Re^3: hang caused by read / readline?
by ikegami (Patriarch) on Sep 04, 2009 at 06:34 UTC
    Use block mode to filter out the NULs first. 
    perl -pe'BEGIN { $/ = \(64*1024); } s/\0+//g' infile | line_reading_sc
+ript.pl

    [download]
[reply]
[d/l]

      (Sorry if this is a repeat, but my last reply doesn't appear to be visible in the main node view.) How could that first part of your code be incorporated into an existing script rather than being issued from the command line? I would like to perform the s/\0+//g substitution on the input file and then save it before moving on to the rest of the script.

[reply]
        my $buf = '';
local $/ = \4096;
while (<>) {
    s/\0+//g;
    $buf .= $_;
    process_line("$1") while $buf =~ s/^(.*)\n//;
}
process_line($buf) if length($buf);

        [download]
[reply]
[d/l]

      Hmmm, how could that first part be incorporated into an existing script rather than being issued from the command line?

[reply]

In Section Seekers of Perl Wisdom