All of the help you have given is greatly appreciated. I'm just a newbie trying to do the best I can. One last question: So I'm trying to put it all together and read three different files and query through the proxy with the values pulled. Basically something like this:
1)use proxy one <value1 for file>
2)to search URL containing keyword<value 1 from 2nd file> + city<value 3 from 3rd file>
I have the for and the while that grabs the keyword and city from their respective files and it work perfectly but I can't seem to figure out how to put the proxy piece in.
Nesting a for in a for doesn't seem to work and with the current code I'm not getting all three values changing as expected. Just the proxy value.
Here's what I have
open(my $fh, '<', 'cities.txt') or die $!;
print $fh;
open(my $prx, '<', 'proxies.txt') or die $!;
print $prx;
mkdir 'Bing', 0755;
mkdir 'Bing/1Parsed/', 0755;
mkdir 'Bing/1Parsed/Html/', 0755;
chomp(my @cities = <$fh>);
close($fh);
chomp(my @prx = <$prx>);
print $prx;
close($prx);
open($fh, '<', 'keywords.txt') or die $!;
for my $city (@cities) {
seek($fh, 0, 0);
while (my $keywords = <$fh>) {
chomp($keywords);
print "$city $keywords\n";
#toggle proxy selection
my $ua = LWP::UserAgent->new;
$ua->agent('Mozilla/8.0');
for my $p (@prx){
print "Current proxy:$p\n";
$ua->proxy(['http'], "http://$p");
# Use this UA/Proxy to fetch something....
my $xml1 = $link1 . $city ."+". $keywords . $link2 .
+$city ."+". $keywords . $link3;
#my $xmla = $link3 . $row . ".com";
#my $xmlx = $link4 . $row;
my $filename1 = "Bing/".($city)."_". ($keywords) . ".
+html";
open my $fh1, ">", $filename1 or die("Could not open
+file. $!");
my $xml2 = get $xml1;
print $xml1;
print "\n";
print $fh1 $xml2;
}
}
}
When it's late I'm often not as clear headed ass I'd like to be as evidenced by my posts tonight! |