I'm having problems with the following sub routine, when it gets going my CPU usage shoots up to 100% for however long it takes to run, sometimes over an hour. I was just wondering why it was so inefficient??

I'm using Win XP Pro, and Active Perl 5.8.

Initially the routine opens up a small txt file which contains a file path, it checks if the file exists, if it does then it carries on, if not it writes the whole of the data file to $data which is then spat out in an email.

Then it is designed to open up the two text files (new data and old data) containing data in the following format: 
101210::20041009::YYYYYYY::::1096930741::_1_
101210::20041016::YYYYYYY::::1096930741::_1_
101210::20041023::NNNNNNN::::1096930741::_1_
101210::20041030::NNNNNNN::::1096930741::_1_
101210::20041106::NNNNNNN::::1096930741::_1_
101212::20041009::NNNNNNN::::1096930741::_2_
101212::20041016::NNNNNNN::::1096930741::_2_
101212::20041023::NNNNNNN::::1096930741::_2_
101212::20041030::YYYYYYY::::1096930741::_2_
101212::20041106::YYYYYYY::::1096930741::_2_

[download]
It writes the two sets of data into an array each split by the endline character. Then goes through line by line and compares them using the 'cid' and 'date' sections at the beginning of each line i.e. 101212::20041106::, and then checks whether the 'avs' i.e. YYYYYYY is the same in each file where the 'cid' and 'date' are the same. Hope that makes sense! 
sub compare_avs  {
    
    # find the name of the file created at last running

open(DIFF_OLD, $path.$agent."/AVS/DIFF_OLD.txt") || die "ERROR: Unable
+ to open ".$agent." - DIFF_OLD_FILE.txt";
    $DIFF_OLD = <DIFF_OLD>;
    close DIFF_OLD;

        # check if file exists if not, write contents of new file to $
+data
    
    if ($DIFF_OLD eq "")  {
        print "DIFF_OLD file - Does not exist\n"; 
        open(DIFF, "$EMAILfile") || die "ERROR: Unable to open ".$agen
+t." - ".$EMAILfile;
        while(my @AVSn = split/\n/, <DIFF>) {
            foreach $AVn(@AVSn) {
                $data = $data.$AVn."\n";
            }
        }
        close DIFF;
    }  else  {
        print "DIFF_OLD file - ".$DIFF_OLD."\n"; sleep(5);
        
                # write contents of old file to an array

        open(CHECK, $path.$agent."/AVS/".$DIFF_OLD) || die  "ERROR: Un
+able to open ".$agent." - ".$DIFF_OLD;
        @AVSo = <CHECK>;
        close CHECK;
        
        # write contents of new file to array and loop through it

                open(DIFF, "$EMAILfile") || die "ERROR: Unable to open
+ ".$agent." - ".$EMAILfile;
        while(my @AVSn = split/\n/, <DIFF>) {
            foreach $AVn(@AVSn) {
                $AVSchecked = 0;
                ($SIDn,$DATEn,$AVSn,$NOTHINGn,$EPOCHDATEn,$SERIALn) = 
+split/::/,$AVn;
                        # compare contents of old list with new list

                                        foreach $AVo(@AVSo)  {
                    $AVo =~ s/\n|\r//g;
                    ($SIDo,$DATEo,$AVSo) = split/::/,$AVo;
                    if ($SIDn eq $SIDo && $DATEn eq $DATEo && $AVSn ne
+ $AVSo)  {
                        $data = $data.$SIDn."::".$DATEn."::".$AVSn."::
+::".$EPOCHDATEn."::".$SERIALn."\n";
                        print $SIDn."::".$DATEn."::".$AVSn."::::".$EPO
+CHDATEn."::".$SERIALn."\n";
                        $AVSchecked = 1;
                    }  elsif  ($SIDn eq $SIDo && $DATEn eq $DATEo && $
+AVSn eq $AVSo)  {
                        $AVSchecked = 1;
                    }
                }
                if ($AVSchecked == 0)  {
                    $data = $data.$SIDn."::".$DATEn."::".$AVSn."::::".
+$EPOCHDATEn."::".$SERIALn."\n";
                    print $SIDn."::".$DATEn."::".$AVSn."::::".$EPOCHDA
+TEn."::".$SERIALn."\n";
                }
            }
        }
        close DIFF;
    }
    
    $DIFF_OLD_filex = $path.$agent."/AVS/DIFF_OLD.txt";
    open(DIFF_OLD,">".$path.$agent."/AVS/DIFF_OLD.txt") || die "ERROR:
+ Unable to open ".$agent." - DIFF_OLD_FILE.txt to update";
    print DIFF_OLD $agent.$$.".AVS";
    close DIFF_OLD;
    
}

[download]

In reply to Array loop is very inefficient, 100% CPU usage by waiterm

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.