forked die() in eval block has counterintuitive behavior?

rlucas has asked for the wisdom of the Perl Monks concerning the following question:

A die() occurring within a forked child process will get caught by a containing eval block ~~in the parent~~ that expects to happen only once. Normally, one might expect a programmer to account for a fork() in his error handling. However, since a number of commonly-used CPAN modules use fork() or other forking (e.g. via IPC::OpenX or the open("|-") idiom) without mentioning this in the docs, it is easy to get bitten. Did I miss the memo that says we have to check whether a new process was spawned after every eval block?

While implementing a multistep database transaction, I decided to use an idiom I learned in Java: begin transaction , try/catch block, and either commit or rollback based upon the presence of an exception.

In Perl, it was roughly:

eval {
  $dbh->begin();
  # stuff...
  $mime_entity_instance->send();
}
if ($@) {
  $dbh->rollback();
}
else {
  $dbh->commit();
}
[download]

The MIME::Entity instance is a multipart email message, and uses Mail::Mailer to send itself. Unbeknownst to me, Mail::Mailer forks a child process, and when I used taint mode, the PATH was dirty and caused the Mail::Mailer child process exec() to fail. I had a hell of a time figuring out why my code was trying to both commit and rollback.

I've submitted a patch to Mail::Mailer, at http://rt.cpan.org/NoAuth/Bug.html?id=12890

While I can locally solve this in a number of ways, the larger question this brings up is if it is EVER OK to call outside code within an eval block. I can't imagine that everybody, everywhere, checks process IDs before continuing after an eval failure.

You can reproduce the general problem yourself:

#!/usr/bin/perl -w

use strict;

my $num = '123'; # an external database ID, for example
warn "(PID $$) Beginning transaction #$num ";
eval {
    my $pid = fork();
    if ($pid) {
        warn "\t(the parent does OK)";
    }
    else {
        warn "\t(the child fails)";
        die "Child process failure";
    }
};
if ($@) {
    warn "(PID $$) Rolling back transaction #$num: died in eval with $
+@";
}
else {
    warn "(PID $$) Committing transaction #$num";
}
[download]

Thoughts from the monks? How generally should one deal with this problem?

Update: Grr, I thought my patch had solved the problem -- however, if you are using it in a database environment, the fact that the code exits in the child seems to be calling DESTROY on objects that have external effects (e.g. turning AutoCommit back on in the database). So, problem NOT solved for those of you looking to use Mail::Mailer and derivatives in a transactional eval block.

Comment on forked die() in eval block has counterintuitive behavior? Select or Download Code

Replies are listed 'Best First'.
Re: forked die() in eval block has counterintuitive behavior? by mugwumpjism (Hermit) on May 24, 2005 at 01:17 UTC
The essence of your complaint is correct. `eval { }` does not know anything about fork. After you fork, both processes continue. Basically this comes down to people not being thorough enough when they're spawning external programs, and/or not passing through error return values from child processes as exceptions correctly. The following constructs should be safe, and libraries using them should raise errors trappable with `eval { }`: system("foo") == 0 or die "foo failed; rc=$?"; defined(my $pid = open FOO, "command\|") or die "exec failed; $!"; # ... do something with FOO ... close FOO; ($? == 0) or die "sub-process `command' failed; rc=$?"; [download] In general, detecting all nature of errors is outside of the scope of `eval { }`. As far as Perl is concerned, MIME::Lite successfully returned without noticing its sub-process failed, so there was no trappable error. $h=$ENV{HOME};my@q=split/\n\n/,`cat $h/.quotes`;$s="$h/." ."signature";$t=`cat $s`;print$t,"\n",$q[rand($#q)],"\n"; [download]	[reply] [d/l] [select]
Re: forked die() in eval block has counterintuitive behavior? by samtregar (Abbot) on May 23, 2005 at 23:19 UTC
A die() occurring within a forked child process will get caught by a containing eval block in the parent. This is incorrect. I ran your test program and I didn't see anything to idicate that you are correct. Here's the output I saw: `(PID 20379) Beginning transaction #123 at foo.pl line 6. (the child fails) at foo.pl line 13. (PID 20380) Rolling back transaction #123: died in eval with Child pro +cess failure at foo.pl line 14. (the parent does OK) at foo.pl line 10. (PID 20379) Committing transaction #123 at foo.pl line 21.` [download] Clearly the eval{} is catching the die() within the child, not the parent. However, the general thrust of your post is quite correct: it is very bad for a module to fork() and not document that fact. Debugging a fork gone wrong can be very hard, particularly if you don't know a fork() might have happened! -sam	[reply] [d/l]
Re^2: forked die() in eval block has counterintuitive behavior? by rlucas (Scribe) on May 24, 2005 at 01:31 UTC
Yes. I misspoke; as the PIDs clearly show (and as my intention in including them was to indicate), the "catch" of the die() is in the child process. An unexpected child die() gets caught by a containing eval block that thinks it's just dealing with the parent. The overall problem, then, is as you state -- not knowing that a fork() has happened. Is there a general way, then, to deal with this in evals? It seems that the casualness with which fork()s are issued in various places makes this problem endemic to the idiom of Perl...	[reply]
Re^3: forked die() in eval block has counterintuitive behavior? by samtregar (Abbot) on May 24, 2005 at 02:18 UTC
An unexpected child die() gets caught by a containing eval block that thinks it's just dealing with the parent. I don't think this problem is specific to eval{}. Any code which is going to fork needs to be careful about which code is going to run in the parent and which in the child. It's just a fact of Unix programming, not really a problem with Perl specifically. -sam	[reply]
Re: forked die() in eval block has counterintuitive behavior? by Zaxo (Archbishop) on May 23, 2005 at 23:36 UTC
Is this on win32? You may have found a weakness in the fork emulation. You may find it documented in perlwin32. With a proper fork, nothing that happens in the child can affect the parent without explicit IPC. Do you have a `$SIG{CHLD}` handler set? See perlipc. After Compline, Zaxo	[reply]
Re^2: forked die() in eval block has counterintuitive behavior? by rlucas (Scribe) on May 24, 2005 at 13:03 UTC
As mentioned to samtregar above, I misrepresented that the die was caught "by" the parent -- in fact, it was caught in the child, by code that never was intended to have been forked. This is all on Debian Sarge, with Perl 5.8.4 and threads enabled. But I don't think any of it is platform-specific: once I understood what the code was underneath, it became clear that things were going by the book.	[reply]
Re: forked die() in eval block has counterintuitive behavior? by sgifford (Prior) on May 24, 2005 at 02:22 UTC
Huh, that's really interesting. I think the conclusion has to be that it's a bug in the module, but I bet it's a bug in most modules using `fork`. I think the short-term workaround is to check the PID, as you've done, and the long-term fix is to evangelize module authors to use `exit` instead of `die` to fail.	[reply] [d/l] [select]
Re^2: forked die() in eval block has counterintuitive behavior? by rlucas (Scribe) on May 24, 2005 at 14:42 UTC
sgfifford, that's really the heart of my question -- is it incumbent upon the module user or author to check for and work around this? I normally have little sympathy for module users -- if you are willing to accept the benefit of someone's code, you should accept the burden of playing by its rules -- but in this case, I was using MIME::Entity, which used Mail::Internet, which used Mail::Mailer -- nowhere in MIME::Entity's API was it clear that a fork would be happening. Based upon what I've heard in this node and elsewhere due to this experience, I now beleive that CPAN modules that use fork(), open-to-pipe, or similar calls MUST account for the possibility of being called inside an eval -- or else document that the code MUST NOT be called inside an eval without checking for a fork gone wrong. I don't believe this is especially burdensome to CPAN authors, because the patch to Mail::Mailer that solved this problem was only a few lines of code (most of which were splitting out a logic condition into an if/then/else). Am I on the wrong track here?	[reply]
Re^3: forked die() in eval block has counterintuitive behavior? by sgifford (Prior) on May 24, 2005 at 15:05 UTC
No, I think you're right, though if you plan to find and fix every `fork`ing bug in CPAN you certainly have your work cut out for you. And regardless, defensive programming would mandate that you check for the `fork` condition in your calling code. It would be interesting to take this to the `perl5-porters` list and see their take on it.	[reply] [d/l] [select]
Re: forked die() in eval block has counterintuitive behavior? by szabgab (Priest) on May 24, 2005 at 11:19 UTC
Interesting problem and I think you just helped finding the bug I was trying to catch for the past 3 days..... I think putting an eval block within the child code and the exit-ing instead of die-ing will solve the problem. At least in the example you gave: `#!/usr/bin/perl -w use strict; my $num = '123'; # an external database ID, for example warn "(PID $$) Beginning transaction #$num "; eval { my $pid = fork(); if ($pid) { warn "\t(PID $$) the parent does OK"; } else { eval { warn "\t(PID $$) the child fails"; die "\t(PID $$) Child process failure"; }; warn $@ if $@; exit; } }; if ($@) { warn "(PID $$) Rolling back transaction #$num: died in eval with $ +@"; } else { warn "(PID $$) Committing transaction #$num"; }` [download]	[reply] [d/l]
Re^2: forked die() in eval block has counterintuitive behavior? by rlucas (Scribe) on May 24, 2005 at 12:55 UTC
Yes -- when you attack the problem, you may want to examine my patch to Mail::Mailer, which patch fixes the problem by doing two things: Eval the child code (which ideally performs and exec and hands off the baton to sendmail/whatever); if the child code dies, warn the error and exit(1); So that someone using Mail::Mailer has a way of knowing it went wrong, check the return value from closing the filehandle that had been opened on the child process; die here (in the parent!) if a problem. But heavens to Betsy! this is a pernicious problem. I had originally suspected that fork() was being called, so I overrode CORE::GLOBAL::fork with a debugging message and stack trace -- all to no avail, since it was the open("\|-") style of forking that was being used in Mail::Mailer.	[reply]
Re: forked die() in eval block has counterintuitive behavior? by astroboy (Chaplain) on May 24, 2005 at 00:47 UTC
If you're not wedded to Mail::Mailer, what not use MIME::Lite to send your attachments? I've not used Mail::Mailer, but from browsing the doco it seems that MIME::Lite looks easier to use....	[reply]