in reply to recursive loop and performace issue in find::file

Ouch! Look at the following: 

foreach $input(@inputarray) 
{
....

   @inputarray = (@newarraypreceding , @newarrayfollowing);
...

[download]
Never try to remove elements from the array you are currently looping through ... unless you are looking for trouble ;-) (or until you know exactly what you do!) This is probably the reason for your infinite loop.

Regarding speed: you do a lot of array-copying (splicing and putting together the arrays). Be aware that copying arrays is potentially slow ...

With your program you seem to want to create a list of filenames and check if they are existing in the filesystem. A typical approach would be to use a hash for the list of filenames (use the filename as keys!). For each filename you find in the file-system you would then check if a corresponding entry exists in the hash - and either mark it there or remove it.

If you really need the result in an array (which I doubt), you could create an array out of the hash once at the end.

HTH, Rata

btw.: looking up something in a hash is typically much faster than looking it up in an array...

Replies are listed 'Best First'.
Re^2: recursive loop and performace issue in find::file
by Limbic~Region (Chancellor) on Apr 26, 2011 at 18:51 UTC
    Ratazong,
    Never try to remove elements from the array you are currently looping through ... unless you are looking for trouble ;-) (or until you know exactly what you do!) This is probably the reason for your infinite loop.

    Indeed. I think it would have been ok to go on and explain a couple of the situations where it is ok and how to address it: 

    for my $idx (reverse 0 .. $#array) {
    my $item = $array[$idx];
    if (some_test($item)) {
        splice(@array, $idx, 1); # remove the element
    }
}

    [download]

    In the above case, it is ok to remove items because of the order they are being processed. Here is another example: 

    while (@work) {
    my $item = shift/pop @work;
    # Possibly push/unshift items onto @work
}

    [download]

    In the above case we aren't looping over the array, we are using it as a boolean. When all the items have been removed the loop exists. This is useful when implementing your own stack/queue based solution and avoiding recursion.

    I agree with your advice. I am just frustrated when people tell me don't do that unless you know what you are doing. I try to give them examples when it is ok, explain why or examples when it isn't ok and explain why not. I hope you don't mind that I took your advice a little further.

    Cheers - L~R

[reply]
[d/l]
[select]

      Limbic~Region,

      I agree that the phrase don't do this unless you know more is frustrating for some people.

      According to my experience there are three steps when learning (not only programming languages!):

      1. learn (simple) rules
      2. learn the reason for each rule
      3. learn when to break a rule - and what risk is involved
      Based on the code provided by the OP, I assumed that the adequate answer would be on step 1 ... and that further details would be more distracting than helpful. Creating an additional node with further information and excellent examples (as you did) is a very good improvement - especially for other readers which already possess more knowledge/insight. So I certainly don't mind your comments :-)

      Have a nice day! Rata
[reply]

In Section Seekers of Perl Wisdom