As you are already using LWP::Simple you can use that module's head() function to retrieve the size of the file:

my %hash;
while ( my $token = $p->get_tag("a") ) {
    if ( my $href = $token->[1]{'href'} ) {
        # may need to prefix domain to $href
        my ($type, $length, $mod, $exp, $server) = head($href);
        $hash{$href} = $length;
    }
}
[download]