in reply to File I/O Slow Down

dws' observation not withstanding, there are couple of other 'peculiarities' with your code.

The first is this nested loop

foreach( keys %invalid ) { undef @tmp; @tmp = split $invalid{$_}, /\t/; while( @tmp ) { next if $_ =~ m/$fields[$_]/; } }

Apart from chewing a potentially large number of cycles, this doing nothing that I can see? I think I know what you are trying to do, but next will repeat the nearest enclosing loop.

If you want to skip to the next line from INPUT_FEED, then you would need to use the next LABEL; form.

RECORD: while( <INPUT_FEED> ) { .... foreach my $invalid (keys %invalid ) { ... while( @tmp ) { next RECORD if ...; } }

Then there is this statement

next if $_ = /$fields[$_]/;

You have two references to $_ in that if clause, and I think you are expecting them to refer to different things? They won't!

You have #use strict; at the top of your code. I cannot recommend enough that you uncomment that and add -w or use warnings. And then shut teh compiler up by correcting each problem is finds and yells about.

The compilers attention to detail is pretty impecable. If it tells you there is something wrong, it is usually right:).

Also,

$PostCodeStrings{$fields[5]} = $PostCodeStrings{$fields[5]} . $fields[4] . "|" . $fields[5] . "|" . $fields[3] . "|" . $fields[2] . "|" . $fields[1] . "|" . $fields[0] . "\n";

You could save yourself a lot of typing by using  $x .= ... rather than $x = $x . ..... And even more by using join and an array slice on @fields for the rest of the statement. This should be equivalent.

$PostCodeStrings{$fields[5]} .= join( '|', @fields[4,5,3,2,1,0] ) . "\n";

Examine what is said, not who speaks.
"Efficiency is intelligent laziness." -David Dunham
"When I'm working on a problem, I never think about beauty. I think only how to solve the problem. But when I have finished, if the solution is not beautiful, I know it is wrong." -Richard Buckminster Fuller
If I understand your problem, I can solve it! Of course, the same can be said for you.

Replies are listed 'Best First'.
Re: Re: File I/O Slow Down
by agentsim (Initiate) on Aug 05, 2003 at 12:46 UTC
    Thanks for the extra pointer -- I fixed those myself once the split(...) stuff was pointed out.

    One thing I did notice that I think is of note. I did as you suggest and used the join command instead of typing the whole thing out. My execution time fell from ~600 seconds to ~60 seconds.

    Is that because perl do not have to copy the string to memory temporarily and then reassign it when you use join?