sitnalta has asked for the wisdom of the Perl Monks concerning the following question:

Hello, I am currently attempting to learn perl and would like some help or examples to get me out of this rut. I have three files, File1 is local while File2-1 is accessed via ssh host2-1 and File2-2 accessed via ssh host2-2.
I need to search File1 for a keyword, when this keyword is found on any number of lines it will take $commonnumber in that line then it will need will to look in to both File2-1 and File2-2 for this same number and print out the $uniquenumber associated. File2-1 and File2-2 are not identicle, so it may find $uniquenumber on either or both File2-1 or File2-2.
Lets say the information in each file is in the following format: File1: $message, $commonnumber File2-1: $uniquenumber, $commonnumber File2-2: $uniquenumber, $commonnumber
Now I can search the local file, find the line I want, then output it in the form I would like but I cannot figure out howto look in File2-1 and File2-2 and grab $uniquenumber. I tried using the following without any success to get this information:
my @whatever01=system('ssh somehost "grep $commonnumber File2-2"');
This is what I have so far, that only prints info i want in File1, I have no idea where to go from here to print info from two different files to the terminal screen. This info is probley pointless for me to post here but oh well.
#!/usr/bin/perl -w use strict; open(File1, "File1"); while (<File1>) { if (/@ARGV/) { my @atlmmg01=split(/:/); my $commonnumber=@atlmmg01[17]; print "----------------------------------------------- +---------- ----\n"; print "@atlmmg01[2]\n"; print "@atlmmg01[17,18]\n"; print "@atlmmg01[26]\n"; } }

Replies are listed 'Best First'.
Re: Searching in remote files over SSH
by Tanktalus (Canon) on Nov 21, 2006 at 21:56 UTC

    Are the files small? If so, I'd try loading them into memory before your loop through File1 - especially if you're going to do repeated searches:

    my @file21 = `ssh host2-1 "cat File2-1"`; my @file22 = `ssh host2-2 "cat File2-2"`;
    Then you can use the grep internal function to look for what you want.

    (This is one of the few times I would use the backticks for running programs...)

      The files are rather large from 30 - 500 megs.

        30 meg? Bah, that's small. 500 meg? Meh, not small, but not large, either.

        However, if you're concerned, I would still copy the files locally, just to cut down on network transmission from repeated searches. If that's not something you want to do, you could create a perl script that did all the searches based on File1, and run that over ssh.

        Finally, you could just create a perl script that you put on to each server that you could interact with, and, using IPC::Open2, you could send it queries to do on the remote machine, and it would send replies back...

        TIMTOWTDI isn't just a perl maxim, it applies to all of unix ;-)