Hi Scott_J, and welcome to the monastery.

You might be interested in How do I parse links out of a web page, and no doubt a super search will turn up some more material.

update... removed the regexp I put here as it was incorrect... back in a sec with a better one... andy.

continued...

Basically you'd probably be better parsing the HTML, but if you want to avoid doing that then you could use a regexp like this: 

#!/usr/bin/perl -w
use strict;

use LWP::Simple;
my $html = get('http://www.bbc.co.uk/');

while ($html =~ m|href\s*=\s*"((?:[^/]+://[^"/]+)?)/?([^"]+)"|gi) {

        print "$1, $2 \n";

}

[download]
which seems to work OK.

However, there's a couple of drawbacks:
- it's quite hard to read,
- it'll fail in certain situations, e.g. if the page contains quoted html as part of the actual page text, and probably in lots of other situations I haven't thought of.

But... if this is just a quick and dirty hack, and reliability isn't a big issue, then the above regexp may do what you need.

All the best,
Andy.

update again... re-read the question and found you also wanted the link text... hold on a mo...

continued... Try this:

while ($html =~ m|href\s*=\s*"((?:[^/]+://[^"/]+)?)/?([^"]+)"\s*>(.*?)
+</A>|gi) {

        print "$1, $2, $3 \n";

}

[download]

In reply to Re: Extracting href's by andye
in thread Extracting href's by Scott_J

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.