Another good reason to use unlink() is to avoid the chance that somebody will subvert your script by slipping an executable named rm into your path.

You could, of course, mitagate this by writing

system("/bin/rm", $ERRORFILE) if -e $ERRORFILE;

Notice also that I reworked your use of system() to pass two arguments rather than one. This avoids the often unavoidable, albeit small, overhead of having system() check for shell metacharacters.

Better, I think, to write

unlink($ERRORFILE) or die "$ERRORFILE: $!"
    if -e $ERRORFILE;