Urbs has asked for the wisdom of the Perl Monks concerning the following question:

Good evening Monks, I have not been able to solve this riddle in about 8 hours either online or with The Camel, but I am so close.. I have two files that I have to compare. The first is a list of clients stored in clientlist.txt. The second is a huge list of all backed up clients with things like backup times (one line with each client and extra info seperated by commas)named client_last_backup.csv. All that I want to do is open the list of clients and the "clb.csv" and whenever a client in the client list is found in the csv, write that line to a new csv. I can only my loop to run once but the clients will all print if a throw a print $client just after the 2nd while. Here is my code: 
open(clients, "<clientlist.txt") || die "Can't open clients: $!\n";
open(file_compare, "<client_last_backup.csv") || die "Can't open file_
+compare: $!\n";
open(CSV_OUT, "> lastbackup.csv") || die "Unable to open lastbackup.cs
+v: $!";
print CSV_OUT "SERVER,CLIENT,LAST SUCCESSFUL FULL,LAST SUCCESSFUL INCR
+,LAST ATTEMPT\n";
my @clients = "";
LOOKUP: while(<clients>)
{
    @clients = $_;    
$client = shift @clients; 
    chomp $client;


    CLIENTS: while (<file_compare> )
    {
        if (m/$client/)
        {        
        print CSV_OUT "$_";
    }
#print CSV_OUT "$client";    
}

[download]
Thank you so much for having a look! Urbs

Replies are listed 'Best First'.
Re: Newbie can't figure out looping a while (if)
by GrandFather (Saint) on Apr 18, 2008 at 01:29 UTC

    Your immediate problem is that you read all of file_compare in the first iteration of the outer loop while handling the first clients line, but don't reset to the start of the file_compare files for the next iteration.

    However, that is a really bad way to do it. You end up reading the file_compare file right through for every line of the clients file which likely to be very slow! Much better is to build a list of clients on a pass through the client list file, then use that list in a separate loop for processing the log file. Consider:

    use strict;
use warnings;

my $clientList = <<DATA;
fred
bob
DATA

my $backupData = <<DATA;
joe,2,3,wilma
fred,1,3,wilma
bob,2,3,freda
freda,7,2,fred
DATA

my $outputData;

open (my $clientsFile, '<', \$clientList) || die "Can't open clients: 
+$!\n";
my @clients = <$clientsFile>;
close $clientsFile;
chomp @clients;

my $match = '\b(?:' . (join '|', @clients) . ')\b';

open (file_compare, '<', \$backupData)
    || die "Can't open file_compare: $!\n";
open (CSV_OUT, '>', \$outputData) || die "Unable to open lastbackup.cs
+v: $!";
print CSV_OUT "SERVER,CLIENT,LAST SUCCESSFUL FULL,LAST SUCCESSFUL INCR
+,LAST ATTEMPT\n";

while (<file_compare>) {
    if (m/$match/) {
        print CSV_OUT "$_";
    }
}

close CSV_OUT;
close file_compare;
print $outputData;

    [download]

    Prints:

    SERVER,CLIENT,LAST SUCCESSFUL FULL,LAST SUCCESSFUL INCR,LAST ATTEMPT
fred,1,3,wilma
bob,2,3,freda
freda,7,2,fred

    [download]
    Perl is environmentally friendly - it saves trees
[reply]
[d/l]
[select]
      Monks, The Seek option did exactly what I needed. Thank you so much!! One more thing that I understand now. The entire scripts processed 500 client names through about 5000 records in four or five seconds.

      I plan on hanging around and reading more posts so that I can learn even more.

      Thanks again!!

      Urbs

[reply]
Re: Newbie can't figure out looping a while (if)
by Cody Pendant (Prior) on Apr 18, 2008 at 01:15 UTC
    This is a glaring error, I would have thought: 

    @clients = $_;
$client = shift @clients;

    [download]

    You can't just magically make an array out of a string, even if the string is a comma-separated list! You're making a one-item array, then shifting that one item off again into $client. So you might as well just have $client = $_;.


    Nobody says perl looks like line-noise any more
    kids today don't know what line-noise IS ...
[reply]
[d/l]
[select]
Re: Newbie can't figure out looping a while (if)
by ikegami (Patriarch) on Apr 18, 2008 at 01:15 UTC

    while (<file_compare>) is reads from the current position in the file until the end of the file. For the second pass of the outer loop, the current position of the file is at its end, as left there by the first pass of the outer loop.

    Solution: Seek to the start of file_compare before the inner loop.

[reply]
[d/l]

Back to Seekers of Perl Wisdom