in reply to WGET equivalent

wget saves the page it loads to a file, while `` expects it to be output to stdout. Try `wget -q -O - $urlname`, -q telling it not to write information out to stderr and -O - to write the fetched webpage to stdout (- usually means stdout to unixy utilities). Or `curl -s $urlname`.

Replies are listed 'Best First'.
Re^2: WGET equivalent
by downer (Monk) on Feb 01, 2008 at 19:12 UTC
    I moved this to a different server in which I have root access. However, there are still several problems. I am now using LWP. here is my code

    #!/usr/local/bin/perl -w use strict; use CGI qw(:standard); use lib qw(/usr/lib/perl5/vendor_perl/5.8.6/LWP); use LWP::UserAgent; use CGI::Carp qw(warningsToBrowser fatalsToBrowser); my $ua = LWP::UserAgent->new; $ua->timeout(10); $ua->agent('Mozilla/5.0'); print header(), start_html(); if( param() ){ my $page_num = param("page_num"); open(fd_s, "./random_sample") or &dienice("can't open input file f +or read: $ !"); my $line = 0; my $page = 0; my $current_page = ""; while(<fd_s>){ $line++; if( $line == $page_num){ last; } } close(fd_s); $current_page = $_; print "<h3>To see what the page $line currently looks like: <a hr +ef=$curren t_page>$_</a> </h3>\n"; $line++; chomp; print "<form action=\"random_200_sites.cgi\" method=\"POST\">\n"; print "<input type=\"hidden\" name=\"url\" value=\"$_\">"; print "<input type=\"hidden\" name=\"page_num\" value=\"$line\">"; print "<h3><input type=\"radio\" name=\"decision\" value=\"spam\" +onclick=\" this.form.submit()\">spam</h3>"; print "<h3><input type=\"radio\" name=\"decision\" value=\"nonSpam +\" onclick =\"this.form.submit()\">nonSpam</h3>"; print "<h3><input type=\"radio\" name=\"decision\" value=\"very ha +rd decide\ " onclick=\"this.form.submit()\">hard to decide</h3>"; print "</form>\n"; open(OUT, ">>web.result") or &dienice("Countn't open result output + file: $! "); my $decision = param("decision"); my $url = param("url"); if($decision eq "spam"){ print OUT "$url 1\n"; }elsif($decision eq "nonSpam"){ print OUT "$url 0\n"; }else{ print OUT "$url 2\n"; } close(OUT); print hr(); my $response = $ua->get($current_page); my $data = $response->content if($response->is_success); if($data){ print "page: $current_page\n$data\ndone", length($data); } }else{ print start_form(); print p("Starts from page ",textfield("page_num")); print p(submit("go")); print end_html; } sub dienice { my($errmsg) = @_; print "<h2>Error</h2>\n"; print "<p>$errmsg</p>\n"; print end_html; exit; }
    there are a few errors which have me mystified. First, all the operations related to the file handle out cause a permission problem. the file in question has everyone with read and write access, so is there any clue what the issue is?

    second, as the program currently stands, there is an issue (a 500 internal server error). perl -p reveals nothing, I am clueless. I've never debugged a CGI before, so can someone give me some hint on how to fix this, and how to address these kinds of problems in general?

    thanks, downer

      The current directory is not what you think. Especially, it's not the directory your script file is in.

      For error messages, look in your webserver error log.

        ok, I seem to have addressed this, LWP::user agent's is_success returns "permission denied" I am not writing anything here, just using a module. Do I need to figure out how to change permission so I can use modules?