You need to quotemeta the string in $p:

$p = $s = "^xxx";
print "$p has $s\n" if $s =~ /\Q$p/; #does print
[download]

(Depending on whether there's something else not to be quoted after the $p (in your real case), you might also want \E to end the quoting, e.g. /\Q$p\E|foo/)