Dear all, I am a real newbie in Perl and from a professor in another university I found a perl code able to count the number of articles in google news mentioning a given keyword by day and country. To be clearer, assume that I want count the number of articles mentioning "hello" in UK day by day. I am using WWW::Mechanize and the code I wrote is the following: 

#!/usr/bin/perl -w

use WWW::Mechanize;  #activate scraper package

# waiting time between observations
$sleep_per_obs = 5;

# www:mechanize agent 
my $agent = new WWW::Mechanize(onerror => undef);

# Safari browser
$agent->agent_alias( 'Mac Safari' );

# target file
my ($target) = 'data_uk.txt'; 

print "Data will save to $target \n";

open ($target, '>', $target) or die ("Sorry, couldn't open $target for
+ writing. \n");


# term to search
$term = "hello";    
print "Search term is ".$term . ".\n";

 for($year=2012;$year<=2014;$year++){
   for($month=01;$month<=12;$month++){
     for($day=01;$day<=31;$day++){
   
$url = "https://www.google.com/search?q=$term&hl=en&gl=uk&authuser=0&s
+a=X&ei=xXJuUp6tMoLcyQGxp4GgCw&source=lnt&cr=countryUK&tbs=cdr%3A1%2Cc
+d_min%3A$month%2F$day%2F$year%2Ccd_max%3A$month%2F$day%2F$year&tbm=nw
+s";
    
# print "URL is ".$url."\n";

$agent->get($url);  
$content = $agent->content();

$content =~ /(\d+),*(\d*) results/;

#assigns results (thousands and hundreds = $1 and $2) to variables
my ($results1, $results2) = ($1, $2);

if ($results2 eq "") {
    $combo = $results1;
}
else  {
    $combo = ($results1*1000+$results2);
}
if ($combo eq ""){
    $combo=0;
}
 print "Number of results $day-$month-$year : $combo \n";
 print $target "$day-$month-$year: $combo \n";
    sleep 5;

}
  }
   }
close $target;

[download]
However, after a while I get stopped and the script returns 0 results, even though I change the waiting time for each request. Does anyone know where I am wrong? Thank you in advance.

In reply to Count of articles Google News by madsoeni

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.