Welcome to the Monastery!

HTML::TokeParser scares you? no problem, Ovid made HTML::TokeParser::Simple that will make your life easier :)

Here it is a little example:

use HTML::TokeParser::Simple;
use strict;
use vars qw( $parser $attributes );

$parser = HTML::TokeParser::Simple->new($ARGV[0]);

while ( my $token = $parser->get_token ) {
  next unless ($token->is_start_tag('a'));
  $attributes $token->return_attr;
  if(exists $attributes->{href}) {
    print $attributes->{href}, "\n";
  }
}
[download]

We also have some nice Tutorials to get you started and many book reviewed at Book Reviews.

Ciao, Valerio

Thanks everyone. I'll have a try at this, this afternoon. :)

You might be interested in How do I parse links out of a web page, and no doubt a super search will turn up some more material.

update... removed the regexp I put here as it was incorrect... back in a sec with a better one... andy.

continued...

Basically you'd probably be better parsing the HTML, but if you want to avoid doing that then you could use a regexp like this:

#!/usr/bin/perl -w
use strict;

use LWP::Simple;
my $html = get('http://www.bbc.co.uk/');

while ($html =~ m|href\s*=\s*"((?:[^/]+://[^"/]+)?)/?([^"]+)"|gi) {

        print "$1, $2 \n";

}
[download]

However, there's a couple of drawbacks:
- it's quite hard to read,
- it'll fail in certain situations, e.g. if the page contains quoted html as part of the actual page text, and probably in lots of other situations I haven't thought of.

But... if this is just a quick and dirty hack, and reliability isn't a big issue, then the above regexp may do what you need.

All the best,
Andy.

update again... re-read the question and found you also wanted the link text... hold on a mo...

continued... Try this:

while ($html =~ m|href\s*=\s*"((?:[^/]+://[^"/]+)?)/?([^"]+)"\s*>(.*?)
+</A>|gi) {

        print "$1, $2, $3 \n";

}
[download]

No offense to andye (still good advice), but parsing HTML with a regexp is a bad idea.

Stick with a tried-and true module. It is far less likely to break on you (usually due to bad HTML, not code) and allows for further learning.

Take for example HTML::TokeParser:

my $content = get($url);
my $ref = \$content;
my $p = HTML::TokeParser->new($ref);
my $token;
while ($token = $p->get_tag("a")) {
    my $href = $token->[1]{href};
    my $text = $p->get_trimmed_text("/a");
    print "$href => $text";
}
## Should work...
[download]

This looks intimitating, and it is. :) However, by learning how to use modules like TokeParser you'll not only get a better handle on what you want to do, but you'll be learning more about Perl, as well.

Also, if you plan on doing this often, I suggest picking up a copy of Perl & LWP. It's a good resource for interacting with websites.

John J Reiser
newrisedesigns.com

Well Scot, i never read the how to get links from web pages before but i have a solution, this code is only helpful if you are trying to get only that link here is my code.

#!/usr/bin/perl -w
use strict;

my @getdata;
while(<DATA>){
    chomp;
    if ( /href=http/ ){
        @getdata=split/\w+\s+|<|>|"|href|=/;
                    
    }
}
print "$getdata[9]";


__DATA__
Go to the BBC's website <a href=http://www.bbc.co.uk">BBC</a> or you 
can visit the inland revenues pages at
<a href="http://www.inlandrevenue.com">Inland Revenue</a>  which will 
give you the information you need
[download]

Hope you can guide you by your self

  use HTML::TreeBuilder;

  # given that the HTML to be parsed is in a variable named $html . . 
+.

  # this line extracts the links from the html:
  my $links = HTML::TreeBuilder->new_from_content( $html )->extract_li
+nks;

  # and this iterates over the list of link data, slightly massaging e
+ach one:
  for ( map { [ $_->[0], ( $_->[0] =~ m#://(.*)# ), $_->[1]->as_text ]
+ } @$links )
  {
    # here are the three variables you wanted:
    my( $url, $address, $text ) = @$_;
  }
[download]

jdporter
The 6th Rule of Perl Club is -- There is no Rule #6.