Have you looked at HTML::LinkExtor? It sounds to be exactly what you want:

HTML::LinkExtor is an HTML parser that extracts links from an HTML document. The HTML::LinkExtor is a subclass of HTML::Parser. This means that the document should be given to the parser by calling the $p->parse() or $p->parse_file() methods.

I've used it successfully in the past with a lot of my parsing code.