in reply to Re^3: Design hints for a file processor
in thread Design hints for a file processor

The file is up to half a gigabyte, I'm not keeping all that in memory. I could split it up by job, I suppose.
  • Comment on Re^4: Design hints for a file processor

Replies are listed 'Best First'.
Re^5: Design hints for a file processor
by moritz (Cardinal) on Jul 07, 2008 at 13:21 UTC
    You don't have to keep it all in memory. My parser uses just one line of lookahead, you can easily refactor the shift @lines; and $lines[0] into subs that work on a file handle.
    { my $line = <DATA>; chomp $line; $line =~ s/^\s+//: # handles '$lines[0]' sub peek { return $line; } # handles 'shift @lines' sub next_line { my $tmp = $line; $line = <DATA>; chomp $line; $lines =~ s/^\s+//; return $tmp; } # handles boolean check for @lines sub is_exhausted { return !defined $line } }

    I didn't test it, but it should work along these lines.

    Instead of nitpicking details, think on the overall architecture and fix small issues for yourself.

      Instead of nitpicking details, think on the overall architecture and fix small issues for yourself.
      Sorry, didn't mean to nitpick, was just thinking out aloud.