I need to use SIG{CHLD} in this case

Then you need to check for EINTR

use Errno qw( EINTR );

for (;;) {
    my $client = $server->accept();
    if (!$client) {
        next if $! == EINTR;
        die("Can't accept: $!\n");
    }

    ...
}
[download]

If I'm using signals anywhere (eg sleep()), do I need to wrap every system call in a loop to retry it??

Well, those that are interruptable, yes. That includes sysread. But sysread should already be in a loop since it's not guaranteed to return as many bytes as you requested.