I've been bit in the neck by open()

I would like to premise that I've some experience now and even though I consider myself to be well far away from being an expert, I would have never expected that it could happen to me to be bit in the neck by open()...

I premise that I always

use strict;
use warnings;
[download]

so that even if I make frequent mistakes, perl rapidly helps me finding them by complaining loudly. In this case however, the program seemed to be fine at all effects except that it silently "didn't work".

Now, this problem I had had to do with a method that properly trimmed down to a bare minimum looked like this:

sub parse {
    my ($self,$hashref)=@_;
    open my $fh, '<:raw', $self->{FILE} or die $!;
    while (<$fh>) {
        chomp;
        $hashref->{$_}++ for split;
    }
}
[download]

Now it happened that I made a trivial mistake in that I called it like a sub (from another method) and like all trivial mistakes it didn't appear to be trivial until I spotted it, which took quite a while.

Conclusion: What happened? "Obviously" that the hashref given as explicit argument was dereferenced and the value corresponding to the key 'FILE' was accessed, which in turn was undef, so that as per perldoc -f open an anonymous (empty) file was opened for reading and the code in the loop was never executed (so that the fatal error it would have triggered for dereferencing an undef value never popped out either!)

Why didn't do the obvious checks (e.g. definedness in the first place) on the filename I was about to open()? Because basically I was already doing them in the method that called the one (corresponding to that) above.

As a side note, I wonder wether perl could emit a warning when one tries to open() an anonymous file for reading (or for writing) only... or are there common situations in which it could be desirable to do so?

Update: I've noticed that the cmt above has been completely misunderstood, hopefully this followup explains better what I really meant.

For a complete but minimal example check this:

#!/usr/bin/perl -l

use strict;
use warnings;

package Foo;

sub new {
    my ($self,$file)=@_;
    bless { FILE => $file }, $self;
}

sub parse {
    my ($self,$hashref)=@_;
    open my $fh, '<:raw', $self->{FILE} or die $!;
    while (<$fh>) {
        chomp;
        $hashref->{$_}++ for split;
    }
}

package main;

my ($file,%hash)=shift;
Foo->new($file)->parse(\%hash);  # right
Foo::parse(%hash);               # wrong
print for keys %hash;

__END__
[download]

Of course the two lines marked with cmts above are very different and it's hard to imagine how one could type mistakingly the latter for the former, but in my real case it was much less so, IMHO.

Comment on I've been bit in the neck by open() Select or Download Code

Replies are listed 'Best First'.
Re: I've been bit in the neck by open() by Anonymous Monk on Feb 16, 2005 at 12:42 UTC
As a side note, I wonder wether perl could emit a warning when one tries to open() an anonymous file for reading (or for writing) only... or are there common situations in which it could be desirable to do so? Perl doesn't warn on deliberate features, whether common or not. Perl autovivifies undefined filehandles, and array and hash references, and allows you to do `my $var; $var += 3;` without warning as well. They are all features. That's life in a featurefull language like Perl. You can't expect to have your hand held at all times. Using strict and warnings only give you some guidance, but that doesn't mean any carefree coding will be caugth. Just like wearing seatbelts doesn't mean your car can't drive at unsafe speeds.	[reply]
Re^2: I've been bit in the neck by open() by blazar (Canon) on Feb 16, 2005 at 14:38 UTC
As a side note, I wonder wether perl could emit a warning when one tries to open() an anonymous file for reading (or for writing) only... or are there common situations in which it could be desirable to do so? Perl doesn't warn on deliberate features, whether common or not. Perl autovivifies undefined filehandles, and array and hash references, and allows you to do `my $var; $var += 3;` without warning as well. They are all features. Sorry, but you misunderstood what I meant, or maybe I didn't express myself clearly. I'm not referring to autovivification et similia in general, of which I am aware too and that I consider to be great features myself as well. I am referring to a particular, "special" `open()` feature, and in particular that of opening an an anonymous FH when it is passed `undef` as a third argument with the three-args form of call. You'll notice that the example given in the doc itself is with mode '+>', that it says one can also use '+<' instead, but it gives a warning about it. The cmt I made was about opening such anonymous files in '<' (or '>') mode, which could have a sense if the FH is subsequently duplicated, but which is likely, IMHO, not to be what the average user would want in the first place...	[reply] [d/l] [select]
Re^3: I've been bit in the neck by open() by dragonchild (Archbishop) on Feb 16, 2005 at 14:50 UTC
I am referring to a particular, "special" open() feature, . . . You're complaining that a poorly-known, but well-documented, feature of open() doesn't throw a warning when you accidentally trip over it. Ok ... I can buy that. I think the real discussion needs to be whether or not there should be a warning when opening an anonymous filehandle. That's a fair discussion to have. Unfortunately, you're probably going to be Warnocked if you post to p5p about this. They tend to frown upon making changes that violate the Principle of Least Surprise. You might be able to do something yourself with warnings::register and overriding CORE::open(), but that may be more trouble than it's worth. Being right, does not endow the right to be rude; politeness costs nothing. Being unknowing, is not the same as being stupid. Expressing a contrary opinion, whether to the individual or the group, is more often a sign of deeper thought than of cantankerous belligerence. Do not mistake your goals as the only goals; your opinion as the only opinion; your confidence as correctness. Saying you know better is not the same as explaining you know better.	[reply]
Re^4: I've been bit in the neck by open() by blazar (Canon) on Feb 17, 2005 at 09:57 UTC
Re^2: I've been bit in the neck by open() by RazorbladeBidet (Friar) on Feb 16, 2005 at 13:21 UTC
allows you to do my $var; $var += 3; without warning as well. That's interesting. I would have figured it would warn of an uninitialized variable. This was all I could find: Use of uninitialized value%s (W uninitialized) An undefined value was used as if it were already defined. It was interpreted as a "" or a 0, but maybe it was a mistake. To suppress this warning assign a defined value to your variables. To help you figure out what was undefined, perl tells you what operation you used the undefined value in. Note, however, that perl optimizes your program and the operation displayed in the warning may not necessarily appear literally in your program. For example, "that $foo" is usually optimized into "that " . $foo , and the warning will refer to the concatenation (.) operator, even though there is no . in your program. [download] What is the reasoning behind this?	[reply] [d/l]
Re^3: I've been bit in the neck by open() by dragonchild (Archbishop) on Feb 16, 2005 at 13:27 UTC
`sub unique { my %seen; $seen{$_}++ for @_; keys %seen; } my @unique_values = unique( @some_values );` [download] While it's not the best or most efficient way, it is the standard way to extract the unique elements from an array. Do you really want N "uninitialized variable" warnings where N is the number of unique elements in your array? And, that's just one common usage. Being right, does not endow the right to be rude; politeness costs nothing. Being unknowing, is not the same as being stupid. Expressing a contrary opinion, whether to the individual or the group, is more often a sign of deeper thought than of cantankerous belligerence. Do not mistake your goals as the only goals; your opinion as the only opinion; your confidence as correctness. Saying you know better is not the same as explaining you know better.	[reply] [d/l]
Re^4: I've been bit in the neck by open() by RazorbladeBidet (Friar) on Feb 16, 2005 at 13:32 UTC
Re^5: I've been bit in the neck by open() by dragonchild (Archbishop) on Feb 16, 2005 at 13:37 UTC
Some notes below your chosen depth have not been shown here
Re^4: I've been bit in the neck by open() by blazar (Canon) on Feb 16, 2005 at 14:42 UTC
Re^3: I've been bit in the neck by open() by Anonymous Monk on Feb 16, 2005 at 14:54 UTC
What is the reasoning behind this? Programmer convenience. Remember that Perl is there to serve the programmer - not the other way around. Anytime you have to do something for the sole reason of not triggering a warning, you've not been productive. Warnings should not get in the way - the more warnings there are that give false positives, the more likely it is warnings are ignored, or shut off completely. (As any security or ergonomics expert can tell). Wanting to do `+=` (or `.=`) on undefined values is common enough to not have a warning emitted. Beside the example given, here's another: `$sum += $_ for @list;` [download] Note that Perl is smart enough to figure out that having undefined values for `*=` or `/=` is probably a mistake, and it warns then.	[reply] [d/l]