in reply to Reading 60k lines in file

Also add checks for your implied assumptions: does the split succeed and return 5 elements; what's with the open and the 2 gd operations in case of errors?

An example: for the split, you could use a more readable version of e.g. @h=split(/.../, $line) and $#h>=4 or do {warn "illegal line: $line"; next} to bail out on questionable input.

Also of interest for you: is it always the same line / file creating the problem? Missing or bad image file, ....? Which may make one of the gd calls die. Test returns before continue to use them. Try wrapping in eval{} and then testing $@ (e.g. do{warn ...; next} if $@)on calls that can die and warn/skip for the problem line.

HTH, Peter

Replies are listed 'Best First'.
Re^2: Reading 60k lines in file
by b_gsmls (Initiate) on Oct 08, 2009 at 16:24 UTC
    The elements are confirmed to return properly, it works for several thousand images.
    It's always the same line, yes. I have 100 of these files, 33k -60k lines. each one works up to ~4120, breaks. then ~15k more work with no error, then break with error code
[reply]
      The elements are confirmed to return properly

      Not in your provided code.

      it works for several thousand images

      That translates to 'infrequent error', and to 'can process some files successfully'. As do the statements about another 15K ... .

      It's always the same line, yes.

      Same line in the code, or hopefully and VERY helpful in debugging: always the same $line in the INPUT!? (Update: is it always the same image file or not? See Ikegamis comment below as well! A warn "input: $line\n" is a bit verbose but will pinpoint the file in question)

      Please check this and do add code to check the returns esp. from split and gd.

      Update: So you say, it's not the same file triggering the error. Not so nice. Ok. Try Ikegami's code or update your own suitably. If things still fail, then change

      my $gd = $image->resize(160, 120);

      [download]
      my $gd; 
eval{$gd=$image->resize(160, 120)}; 
do{warn "processing error for file: $line - skipping\n"; next} if $@ o
+r not ref $gd;

      [download]

      Similar caution for the 2nd gd call. I'm assuming you've some error + sanity checking by now for the open, split and gd calls.

[reply]
[d/l]
[select]
        NOT the same file. Same line. Thanks for all ur help so far.
[reply]

In Section Seekers of Perl Wisdom