Unless the documentation specifically says how it handles $!, you should think of it as the last thing that failed, somewhere buried deep down in the code you called.

Update:I spoke to soon wrt docs. I noticed that the readline docs do say they set $! on an error, but they don't directly mention what they return. In an example it seems to imply that the return value is undef in that case.

Just to strengthen what bluto says, here's the documentation for $! from perlvar(1):

     $!      If used numerically, yields the current value of the
             C "errno" variable, or in other words, if a system
             or library call fails, it sets this variable.  This
             means that the value of $! is meaningful only
             immediately after a failure:

                 if (open(FH, $filename)) {
                     # Here $! is meaningless.
                     ...
                 } else {
                     # ONLY here is $! meaningful.
                     ...
                     # Already here $! might be meaningless.
                 }
                 # Since here we might have either success or failure,
                 # here $! is meaningless.

             In the above meaningless stands for anything: zero,
             non-zero, "undef".  A successful system or library
             call does not set the variable to zero.

             If used an a string, yields the corresponding system
             error string.  You can assign a number to $! to set
             errno if, for instance, you want "$!" to return the
             string for error n, or you want to set the exit
             value for the die() operator.  (Mnemonic: What just
             went bang?)

             Also see "Error Indicators".
[download]

My problem is than the following. Suppose readline returns undef. That can mean two things, either an error or eof. Now how can I distinguish between the two cases. Perldoc perlfunc says it is enough to clear $! before the call, and check it after it. But if readline can accidentally set $! when it succeeds, isn't it possible that it accidentally sets $! when it means to say it's reached eof?

Even if I can count on $! showing the error after readline returns undef, I have to use

$! = 0;
$line = readline $file;
!defined($line) && $! and die "error readline: $!";
[download]

It would be much simpler to write

$! = 0
$line = readline $file;
$! and die "error readline: $!";
[download]

but if what's happened to me is normal, I can't do that.

Note that with libc, there is at least one function where you must use errno to see if there's an error, but it is enough to check errno there, you don't have to check the return value too. From (libc)Parsing of Integers:

- Function: long int strtol (const char *restrict STRING, char **restrict TAILPTR, int BASE)

[...]

You should not check for errors by examining the return value of `strtol', because the string might be a valid representation of `0l', `LONG_MAX', or `LONG_MIN'. Instead, check whether TAILPTR points to what you expect after the number (e.g. `'\0'' if the string should end after the number). You also need to clear ERRNO before the call and check it afterward, in case there was overflow.

The documentation of gnu libc is not very specific in that sense. From (libc)Checking for Errors:

The initial value of `errno' at program startup is zero. Many library functions are guaranteed to set it to certain nonzero values when they encounter certain kinds of errors. These error conditions are listed for each function. These functions do not change `errno' when they succeed; thus, the value of `errno' after a successful call is not necessarily zero, and you should not use `errno' to determine _whether_ a call failed. The proper way to do that is documented for each function. _If_ the call failed, you can examine `errno'.

Many library functions can set `errno' to a nonzero value as a result of calling other library functions which might fail. You should assume that any library function might alter `errno' when the function returns an error.

Does this mean that a function can change errno even if it succeeds? Of course, one can't state any more specific of such a large library as libc, and after all, perl is not libc so perl might behaive differently. Perlvar does not write anything more specific of $! either.

Does this mean that a function can change errno even if it succeeds?

Unfortunately, the general answer is yes. Some systems seem to handle errno better on success (Solaris?), but they are probably the exception and even then I wouldn't trust them 100%. There is a long history behind errno, most of it caused by poor initial "design" (if you can call it that), and it's definitely not going to change soon. If you check errno, you are signing a contract saying you have read the docs on the _specific_ call you are making. If it doesn't mention preservation of errno on success, chances are good that it probably doesn't.

You are right -- if you want to _really_ check for errors, in general you must jump through hoops with additional ugly code.

Two thoughts.

First, why not use eof to check for an end-of-file condition?

Second, a common way to handle this situation is to close the filehandle after readline returns undef, and see if the close fails. If it does, there was some kind of error handling the file.

Can any of you duplicate this problem? I am using perl, v5.8.2 built for i686-linux.

ActivePerl v5.6.1 on Win2k does not exhibit this problem.

perl v5.8.0 for i386-freebsd on FreeBSD does not exhibit this problem.

In both cases, I also tried inserting the code open $g, "<", "nonexistant"; after the open you already have to force something into $! before you clear it.

However, your observations do not contradict $!'s definition saying that it may contain unless an error is returned. The library is allowed to do something like:

$! = bad handle;
if the handle is bad, return undef;
$! = bad permissions;
if don't have read permissions, return undef;
$! = read error;
if can't read from the handle, return undef;
return line read;
[download]

Quoting from man errno on a Linux box:

Its value is significant only when the call returned an error (usually -1), and a library function that does succeed is allowed to change errno.

There's similar language in Stevens' APUE (p14, section 1.7) that you only look at errno when a failure is indicated. So it's not inconceivable that something in the C layers underneath might diddle errno preemptively and then never clear it (relying on the return code to indicate success).

Update: bleh, left out the important qualifier "when a failure is indicated" above.

According to the discussion of this thread, I think we can agree that you should not check $! when readline returns a defined value.

The good news is that it seems you can trust $! when readline returns undef. Thus, you can most likely follow the example of perlfunc:

$! = 0;
$line = <$file>;
if (!defined($line) && $!)
        { die "error readline: $!"; }
elsif (!defined($line))
        { print "reached eof\n"; }
else
        { $line=~/(.*)/; print qq(read a line: "$1"); }
[download]

I say this because it seems that perl cleares errno when it reads eof, at least starting from version 5.8.1. Try this:

echo $'three\nshort\nlines' > a;
 perl -we 'open $f, "<", "a"; for(0..3) { $!=10000; $l = <$f>; $s = $!
+; warn +(defined($l)?$l=~/(.*)/&&qq/"$1"/:"undef"), " $s\n"; }'
[download]

"three" Bad file descriptor
"short" Unknown error 10000
"lines" Unknown error 10000
undef 
undef
[download]

I've tried the same with perl 5.8.0, and have seen that it does not clear $!. This is because clearing $! is done in the PerlIOUnix_read funciton (perlio.c), and that function has changed between 5.8.0 and 5.8.1 as tye has noted.

I am still not sure whether perl always clears errno on eof or only in some conditions (it seems likely that it happens only with some kind of filehandles), so you shouldn't take granted what I've just said above.

I've done some debugging to find out why I get this behaiviour.

I've compiled a vanilla perl 5.8.5 with -O0 -g as cflags. It gives the same output as I've shown in the OP.

I've found that $! is directly connected to errno, and so I've put a watchpoint to errno, and found that errno is set to EBADF by the PerlIO_fast_gets function:

int
PerlIO_fast_gets(PerlIO *f)
{
    if (PerlIOValid(f) && (PerlIOBase(f)->flags & PERLIO_F_FASTGETS)) 
+{
         PerlIO_funcs *tab = PerlIOBase(f)->tab;

         if (tab)
              return (tab->Set_ptrcnt != NULL);
         SETERRNO(EINVAL, LIB_INVARG);
    }
    else
         SETERRNO(EBADF, SS_IVCHAN); // <----

    return 0;
}
[download]

#0  PerlIO_fast_gets (f=0x817c148) at perlio.c:1721
#1  0x081402e6 in PerlIOBuf_fill (f=0x817ca58) at perlio.c:3576
#2  0x0813da21 in Perl_PerlIO_fill (f=0x817ca58) at perlio.c:1654
#3  0x0813e4ff in PerlIOBase_read (f=0x817ca58, vbuf=0xbfffd657, count
+=1) at perlio.c:2059
#4  0x08140498 in PerlIOBuf_read (f=0x817ca58, vbuf=0xbfffd657, count=
+1) at perlio.c:3624
#5  0x0813d670 in Perl_PerlIO_read (f=0x817ca58, vbuf=0xbfffd657, coun
+t=1) at perlio.c:1564
#6  0x08141f06 in PerlIO_getc (f=0x817ca58) at perlio.c:4771
#7  0x080deb25 in Perl_sv_gets (sv=0x818513c, fp=0x817ca58, append=0) 
+at sv.c:6341
#8  0x080cd19b in Perl_do_readline () at pp_hot.c:1588
#9  0x080c8cbb in Perl_pp_readline () at pp_hot.c:231
#10 0x080b32da in Perl_runops_debug () at dump.c:1442
#11 0x080624c0 in S_run_body (oldscope=1) at perl.c:1924
#12 0x08062053 in perl_run (my_perl=0x8175e70) at perl.c:1843
#13 0x0805e5f4 in main (argc=2, argv=0xbffffa04, env=0xbffffa10) at pe
+rlmain.c:86
[download]

I am not a master of perlio (nor of any other part of perl source), so this still doesn't say much to me, but I hope one of you could help me.