Escapes within double quotes can be tricky.

qr// makes it easy.

$regex = qr/href\s*=\s*".*?"/;
[download]

Also,
while ($line =~ /$regex/) { my $match = $&; ... }
can be better written as
while (my ($match) = $line =~ /($regex)/) { ... }
since it avoids globals and $&, which slows down matches throughout your program.