colonelcrayon has asked for the wisdom of the Perl Monks concerning the following question:

Just as a little project, I'm trying to write a script which prints the latest updates from Kilauea volcano in Hawaii. Here's what I have so far: 
#!/usr/bin/perl

use LWP::Simple;
use HTML::Obliterate qw(extirpate_html);

$url="http://volcano.wr.usgs.gov/hvostatus.php";

$mypage=get($url);

my $q=extirpate_html( $mypage );

open OUT, ">/home/jesse/Desktop/Programs/vol.txt" or die $!;

select OUT;

print $q;

close OUT;

open IN, "</home/jesse/Desktop/Programs/ups.txt" or die $!;
select IN;

foreach $line (<IN>) {
        if ($line =~ /^Last 24 hours/) {
            print $line;
        }
}
close IN;

[download]
This should print the lines describing the last 24 hours of activity - but doesn't. The part which downloads the info and pastes it in a text file works flawlessly. It's clearly the search portion which doesn't work. Any ideas about what's going on?

Replies are listed 'Best First'.
Re: Basic search question
by CountZero (Bishop) on Jan 06, 2008 at 22:08 UTC
    You are working with HTML (even XHTML!) so your best bet is to use one of the many parsing modules available.

    HTML::Parser for example.

    If you look at the source of the web-page you see that the latest update is within <kilauea_update>-tags, so all you have to do is extract the text between these tags. 

    use strict;
use HTML::Parser ();
use LWP::Simple;
my $url="http://volcano.wr.usgs.gov/hvostatus.php";

my $mypage=get($url);

# Create parser object
my $grab_text = 0;

my $p = HTML::Parser->new( api_version => 3,
                        start_h => [\&kilauea_update_tag_start, "tag"]
+,
                        end_h   => [\&kilauea_update_tag_end,   "tag"]
+,
                        text_h  => [\&textgrabber,              "dtext
+" ],
                      );

$p->parse($mypage);

sub kilauea_update_tag_start {
    my $tag = shift;
    return unless $tag eq 'kilauea_update';
    $grab_text = 1;
}

sub kilauea_update_tag_end {
    my $tag = shift;
    return unless $tag eq '/kilauea_update';
    $grab_text = 0;
}

sub textgrabber {
    my $text = shift;
    print $text if $grab_text and $text=~/Activity Summary for last 24
+ hours:/;
}

    [download]
    Of course you may wish to save the the text in a variable or file. That is left as an exercise for the reader.

    Update: The reason (amongst others) why your regex did not work is because the web-page has a lot of spurious linefeeds and other strange white-space embedded.

    CountZero

    A program should be light and agile, its subroutines connected like a string of pearls. The spirit and intent of the program should be retained throughout. There should be neither too little or too much, neither needless loops nor useless variables, neither lack of structure nor overwhelming rigidity." - The Tao of Programming, 4.1 - Geoffrey James

[reply]
[d/l]
[select]
      This worked fine. I hadn't thought of using HTML::Parser (because I solved another problem with the other method, and assumed it would work here). Thanks for the good idea!
[reply]
Re: Basic search question
by Corion (Patriarch) on Jan 06, 2008 at 20:40 UTC

    Could it be because you write to one file but read from another?

[reply]
      And print to a read-only filehandle (select IN;), the very file you're reading from.
[reply]
[d/l]
[select]
        I wasn't printing anything on IN - select IN was a careless error.
[reply]
Re: Basic search question
by hipowls (Curate) on Jan 06, 2008 at 22:02 UTC

    You don't want to use select, at least not for this. Here it is setting the default file handle for print statements and has global effect. It will be clearer and less error prone to write print OUT $line;

    I'd forget about using temporary files and do something like

    foreach my $line ( split /\n/, $q ) {
    print "$line\n" if $line =~ /^Last 24 hours/;
}

    [download]

[reply]
[d/l]
[select]
      Your point regarding select is absolutely correct, but the code didn't work (it was just a cleaner way of doing what I did).
[reply]

Back to Seekers of Perl Wisdom