pkumar.pr has asked for the wisdom of the Perl Monks concerning the following question:

This node falls below the community's threshold of quality. You may see it by logging in.

Replies are listed 'Best First'.
Re: Crawler Help
by afoken (Chancellor) on Jun 30, 2010 at 03:53 UTC

    Welcome at perlmonks.

    Note that perlmonks is not a code writing service. Show us your code, wrapped in <code> tags, and show us what is wrong with that code (error messages, unexpected output). Don't ask us how to write malicious code, we won't help you with that, for obvious reasons.

    Alexander

    --
    Today I will gladly share my knowledge and experience, for there are no sweeter words than "I told you so". ;-)

      Hi, When I try to fetch the page using any browser I get the result whereas when I try to fetch it using code I don't see any content.

      #!usr/bin/perl -w use strict; use WWW::Mechanize; my $ua = WWW::Mechanize->new; my $base_url = 'http://search.justdial.com/srch/search_cwa.php?srch=tr +ue&city=Chennai result=&city=&searchtype=Multicuisine+Restaurants&res +ultType=&area=&new_area=&Stype=category&choices=&intermid=true&docId= +&smsAlert=&emailAlert=&NameAlert=&sinfo=&smsDetail=&img=&img_check=&s +eeAlso=&sms_visible=&sms_button=&Lowcat=&lowest=&return_page=&magic_m +obile=&magic_email=&magic_sinfo=&magic_Detail=&magic_seeAlso=&magic_n +ame=&magic_tel=&login_name=&login_pass=&login_name_review=&login_pass +_review=&img_forget=&screen_name=&login_name1=&login_pass1=&comp=&cid +=&review_div=&ADVSEARCH=&city=Chennai&txtname=&txtadd=&Street=&area=& +Pin=&txt_company=&txtphone=&jdG=&search_type_string_company=&search_t +ype_string='; $ua->cookie_jar(HTTP::Cookies->new()); $ua->default_header('User-Agent' => 'Mozilla/5.0 (Windows; U; Windows +NT 6.1; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/3.0.195.3 +8 Safari/532.0'); open(DATABASE, ">just_dial.html") || die "$!\n"; DATABASE->autoflush(1); $ua->get($base_url); print DATABASE $ua->content();

        Does the site use Javascript? Have you tried visiting and operating the site without Javascript from a browser?