Hello,
i think
HTML::LinkExtor will be a useful tool in your case, and
this old node too.
If you want to update a list of unique links you can store them somehow (plain text, database, storable file..) then you firstly load this cache in the program, building up an hash (keys are unique, so it helps). After you can extract links and update the hash only if key does not
exists. On success write the new copy of the storage.
L*
There are no rules, there are no thumbs..
Reinvent the wheel, then learn The Wheel; may be one day you reinvent one of THE WHEELS.