You have to setsid too, to disassociate from your parent's process group:

#!/usr/bin/perl
use strict;
use warnings;
use POSIX qw(setsid);

# Become a daemon
my $pid=fork;
exit if $pid; # Parent exits here
die "Couldn't fork $!" unless defined($pid);
die "Couldn't start new session $!" unless POSIX::setsid();

# Do your daemon bit...