sunadmn has asked for the wisdom of the Perl Monks concerning the following question:

Good afternoon fellow monks. I have been given a task of spliting my current company and old sister companies DNS. Now I have been able to make a list of company and sister company owned domains names and have a script that will perform a whois on either list, but what I would like to be able to do is grab the STDOUT from the command I run and sort each instance of the whois info for certain name servers and then from that store the domain name that the whois was ran on into theree different files (i.e. ours. theirs, and unknown ). This is where I am getting a bit confused. I have thought of using arrays to do this, but my list is large and I dont want to eat memory since this will be run on a production box. With that said can anyone point me in the best direction to do this?? Below is the current code I have to run the whois on the list I have.

Thanks all
-sunadmn
USE PERL
#!/usr/bin/perl -w


# This script will run a whois on a list of domains names

use strict;

my $domains  = '/var/tmp/clean';
my $cmd      = '/usr/local/libtpp-1.32.02.redhat6/bin/tpp -c /usr/loca
+l/libtpp-1.32.02.redhat6/bin/tpp.db -whois';
my $out      = '/var/tmp/out_put';

open(OUT, ">$out")    || die "Cant creat $out\nReason: $!\n";
open(IN, "$domains")  || die "Cant open $domains to read\nReason: $!\n
+";
my @domain = <IN>;
close(IN);

foreach my $line (@domain) {
  chomp $line;
  print "running whois on $line\n";
  system "$cmd $line";
}
close(OUT);
exit;

[download]

Replies are listed 'Best First'.
Re: Automating whois??
by tcf22 (Priest) on Nov 10, 2003 at 19:35 UTC
    You could use backticks(`) to capture the STDOUT of the nslookup, and then write to the files w/o storing them in an array. Maybe something like this. 
    open(OURS, '>/path/to/ours');
open(THEIRS, '>/path/to/theirs');
open(UNKNOWN, '>/path/to/unknown');

foreach my $line (@domain) {
  chomp $line;
  print "running whois on $line\n";
  my $output = `$cmd $line`;
  if(#ours){
    print OURS "$line\n";
  }elsif(#theirs){
    print THEIRS "$line\n";
  }else{
    print UNKNOWN "$line\n";
  }
}

close(OURS);
close(THEIRS);
close(UNKNOWN);

    [download]

    - Tom

[reply]
[d/l]
      Well I am not sure that will work since the output of the whois is very long and this info will have to be sorted, so I assume to match the data I am lookin for.Below is an example of a whois so you can have a better understanding of what our command output is.

      Read more... (7 kB)
[reply]
[d/l]
        You will need to parse the output from the command to get the data you want out of it. You can either read the entire output into memory with backticks. Or read from the command line by line with open pipes. The results aren't that large as long as you don't keep it all in memory for every domain. How you want to handle the parsing makes more difference.
[reply]
Re: Automating whois??
by shockme (Chaplain) on Nov 10, 2003 at 20:27 UTC
    Well, there's always Net::Whois. This would allow you to grab and sort on the information you need, without worrying with all of the parsing.

    If things get any worse, I'll have to ask you to stop helping me.

[reply]
      I did look at CPAN and found that the Net::Whois mod looks like it would work had it not be for the shear number of domains this will be ran on ( about 10,000 total ) so the issue I run into there is just exact timing. There was an issue sometime back where the whois DB servers we just getting spam'd with requests so in turn the whois companies ( Internic, register, NetSol ) all have put in place wrappers that only allow "X" amount of requests per min/sec. Now with that said I have to use the software provided by my partner ( register.com ) to run my whois. With their system I can run an unlimited number of whois requests without the wrapper taking control, so with that said the common CPAN mods aren't going to work unless anyone knows a catch 22 / loophole.
[reply]
        Do you know what "X" is? or have the time/patience to get close enough? If so, would it be unreasonable to have your script perform only a few at a time? You could catch the time at the beginning of the loop through the domains, and if it is going to fast, just pause the program for a while with a syscall. If you can't have the program running all day, just run it at night and keep track of which domains you have done already.
[reply]
Re: Automating whois??
by ehdonhon (Curate) on Nov 10, 2003 at 22:57 UTC
[reply]

Back to Seekers of Perl Wisdom