in reply to Avoid "duplicate" fetching with LWP
How about:
use Data::Dumper; my @uris = qw( http://www.foo.bar/index.html http://www.foo.bar/index.html#foo http://www.foo.bar/index.html#bar ); my %seen; my @unique_uris = grep !$seen{$_}++, map /^([^?#]+)/, @uris; print Dumper(\@unique_uris);
That would catch everything to the left of a # (anchor) and ? (query) character (if there is one), which I believe is what you want.
Hope that helps.
|
---|