Pulling HTML from a remote site and writing a portion of it to a file on server

Whitey has asked for the wisdom of the Perl Monks concerning the following question:

I am having a problem with a script that gets a page from a remote host, and then spits out a selected portion of code to a text file on my server. Is my syntax correct? My server is giving me INTERNAL SERVER 500 errors. I can run other perl scripts, the acces is correct. Please help me.

#!/usr/bin/perl

use LWP::Simple;
use LWP::UserAgent;

$_ = get ("http://www.bloomberg.com/energy/index.html");
$data =~ /<!---------------PETROLEUM-----------------> (.*) <map name=
+"BbgELogin2">/m;
open (FH,"./file.txt") ||die"$!";
print FH $_;
close FH;
[download]

Comment on Pulling HTML from a remote site and writing a portion of it to a file on server Download Code

Replies are listed 'Best First'.
Re: Pulling HTML from a remote site and writing a portion of it to a file on server by bikeNomad (Priest) on Jun 23, 2001 at 22:39 UTC
You don't say what environment this is working in. Is this a standalone Perl application? Anyway, opening the file for output would help (you're opening it for input). And if this is in a CGI, you might try using CGI::Carp. Also, you're processing $data, but not setting it anywhere. Did you mean to use $_ instead?	[reply]
Re: Re: Pulling HTML from a remote site and writing a portion of it to a file on server by Whitey (Novice) on Jun 23, 2001 at 22:43 UTC
This is running on Unix. It is a script that is called from a cron job.Here is an updated script. Can you offer some help? Nothing is writtten to the file. `#!/usr/bin/perl use strict; use LWP::Simple; use LWP::UserAgent; my $data = get("http://www.cyrusoilandgas.com"); my ($wanted) = $data =~ /<!---------------PETROLEUM----------------->\ +s(.)\s*<map name="BbgELogin2">/s; open (FH,"./file.txt") \|\|die"$!"; print FH $data; close FH;` [download]	[reply] [d/l]
Re: Re: Re: Pulling HTML from a remote site and writing a portion of it to a file on server by bikeNomad (Priest) on Jun 23, 2001 at 22:45 UTC
This is because of your open. Try: `open(FH, ">file.txt") \|\| die "$!\n";` [download] BTW, didn't you post your node twice?	[reply] [d/l]
(jeffa) 3Re: Pulling HTML from a remote site and writing a portion of it to a file on server by jeffa (Bishop) on Jun 23, 2001 at 22:46 UTC
That because you are not opening the file for writing. See (jeffa) Re: Pulling HTML off another site problem. And please don't double post. :) Jeff R-R-R--R-R-R--R-R-R--R-R-R--R-R-R-- L-L--L-L--L-L--L-L--L-L--L-L--L-L--	[reply]
Re: Pulling HTML from a remote site and writing a portion of it to a file on server by Beatnik (Parson) on Jun 23, 2001 at 23:05 UTC
You're not printing a HTTP header. Use CGI.pm to print one... `use CGI qw/:standard/; print header;` [download] Greetz Beatnik ... Quidquid perl dictum sit, altum viditur.	[reply] [d/l]