eleron has asked for the wisdom of the Perl Monks concerning the following question:

Hi folks,
I've stuck with the following problem, I need to close all opened files in the process after forking it. Does anybody knows how can I do it? BTW, is there any way to get list of all opened files?

Replies are listed 'Best First'.
Re: how to close all files
by almut (Canon) on Oct 23, 2008 at 12:50 UTC

    Unfortunately, this is a non-trivial problem. The lsof (list open files) utility may help however (if you're on a platform that has it...).  Or the Perl wrapper Unix::Lsof, written by a fellow monk.

    Update: just a few more words on where this can get tricky.

    Consider the not so untypical scenario of getting a long running forked process to dissociate cleanly in some webserver context, such that the (grand-)parent process may continue normally and serve subsequent requests. For this, it's necessary to close any file handles in the child that the grandparent would otherwise wait for to be closed (and thus block/hang). Now, with a regular CGI program this is still rather simple (by default, closing stdout/stderr is sufficient), but if you, let's say, have a FastCGI environment with persistent DBI/DBD connections to an Oracle database, things can get somewhat more involved. In particular because the respective modules and libraries under the hood may have opened files and sockets on their own, which are not under your immediate control. Also, as practice shows, brute force methods closing all handles may indirectly render the parent process non-functional...

    That said, as long as your code has full control over which files are being opened, I agree it's usually best to just keep track of what needs to be closed (as has already been suggested elsewhere in the thread).

Re: how to close all files
by JavaFan (Canon) on Oct 23, 2008 at 13:25 UTC
    Tricky. Here's a non-portable way to do it.

    First, get a list of file descriptors. On many Unix systems, you can use lsof for that, or look in the /proc filesystem (which will have a different format on different OSses). On my system, I can do:

    my @fds = `ls /proc/$$/fd`; chomp @fds;
    Now you have a list of filedescriptors. Unfortunally, you cannot use Perl's close with just the filedescriptor. But don't despair, Perl does give you access to the system calls. syscall is unknown to many people, but it's powerful. On a Unix system, the following ought to work:
    require 'syscall.ph'; # May have to run h2ph first. for my $fd (@fds) { syscall(&SYS_close, $fd) == -1 or die $!; }
    Frankly, I rather keep track of my filehandles. ;-)
      I think this could not flush the last-written buffer on files... Maybe
      exec env => $^X, $0, '--secret-forked-flag', @ARGV
      would work better, as perl closes all files on exec??
      []s, HTH, Massa (κς,πμ,πλ)
        From perldoc -f fork:
        Beginning with v5.6.0, Perl will attempt to flush all files opened for output before forking the child process, but this may not be supported on some platforms (see perlport). To be safe, you may need to set $| ($AUTOFLUSH in English) or call the "autoflush()" method of "IO::Handle" on any open handles in order to avoid duplicate output.
        Forking cannot easily be replaced with an exec of itself. The process may have done extensive computation for instance. Furthermore, after an exec(), you still have open file handles: STDIN, STDOUT and STDERR.
        Why do you care about flushing the last write? Presumably the reason for the closes is to leave the control of all the open files to the parent.

        As for what JavaFan suggests, keeping track of open file descriptors when you know you might need to close them all at the same time is painful, but probably the wisest. Simply finding the open descriptors and closing them may or may not be a good idea. Some open descriptors may be within the context of included modules, and simply closing the descriptor without invoking the 'normal' close/disconnect/exit method might not produce the expected/desired results.

      syscall() is one way. See Re: how to close all files where BrowserUk uses the core POSIX module to close by descriptor, though. It's probably the more portable choice for this task.
Re: how to close all files
by Fletch (Bishop) on Oct 23, 2008 at 13:09 UTC

    There's always the BFI method: start at 0, create a filehandle for that descriptor, close it, increment and repeat until you reach a maximum (there's probably a way to tickle the maximum possible file descriptor number out of POSIX but I can't recall it offhand and my copy of APUE isn't readily available).

    use constant MAX_DESCRIPTORS => 255; for my $fd_no ( 0 .. MAX_DESCRIPTORS() ) { open( my $fh, ">>&=", $fd_no ) or warn "Can't dup FD $fd_no: $!\n"; close( $fh ); }

    Update: Dur, never mind me. The correct mechanism is POSIX::close as is mentioned below, not dup'ing. MENTAL NOTE: NO POSTING BEFORE CAFFEINE . . .

    The cake is a lie.
    The cake is a lie.
    The cake is a lie.

      I think there's some misunderstanding of what a duplicate filehandle does. What you have is very nice for closing the duplicates. It doesn't close the originals, though. Duplicate file handles to the same descriptor may share lock status, but they still open and close separately.

      I did test this with 255 instead of 20 BTW, but you don't really need to see 255 warnings scroll by. There are fewer than 20 files open here, anyway. Just call it the "arbitrarily large" constant people have been tossing out.

      If you really want to close by descriptor, use POSIX::close as suggested by BrowserUk at Re: how to close all files.

      use strict; use warnings; open ( my $foo, '>&=', 1 ); close( STDOUT ); print STDOUT "foo!\n"; print $foo "foo!\n"; open ( my $bar, '>&=', $foo ); close $foo; print $foo "foo!\n"; print $bar "bar!\n"; open ( my $baz, '>', 'files/baz' ); open ( my $quux, '>&=', $baz ); close $baz; print $quux "quux!\n"; open ( $baz, '>>', 'files/baz' ); open ( $quux, '>>&=', $baz ); close $quux; print $baz "baz!\n"; open ( my $prefork, '>', 'files/prefork.txt' ); if ( my $kidpid = fork ) { my $deadkid = wait(); print $prefork "child was pid $deadkid, which thankfully matches $ +kidpid (we'd be worried if it didn't)\n"; } elsif ( defined $kidpid ) { for ( 0..20 ) { open ( my $postfork, '>>&=', $_ ); close $postfork; print $postfork "Whee!\n"; } print $prefork "Dupes are just copies!\n"; } else { print "I couldn't fork!\n"; }
Re: how to close all files
by mr_mischief (Monsignor) on Oct 23, 2008 at 17:14 UTC
    You can close STDIN, STDOUT, and STDERR by name. As for the rest, a lexical filehandle does not need to be a standalone scalar. An array entry or a hash entry is a scalar, too.

    The sample code assumes you have a subdirectory called "files" in the current directory to keep from polluting your current directory. (I'm planning a Meditations post later about my method of organizing PM tests, examples, benchmarks, and sample data files for them. I'll update this with a ref to that when it's posted).

    First, we have an array. Notice that the files get successfully closed then one receives warnings about the second print to each.

    use warnings; use strict; my @foo; for ( 1..5 ) { open $foo[$_], '>', "files/file$_.txt" or die "Cannot write to file$ +_.txt: $!\n"; } for ( 1..5 ) { print { $foo[$_] } "$_\n"; } for ( 1..5 ) { close ( $foo[$_] ) or die "Cannot close file$_.txt: $!\n"; } for ( 1..5 ) { print { $foo[$_] } "$_\n"; }

    Now we have a hash, which has multiple descriptive handle names grouped in one handy data structure. It closes then tries to print to the files just as the array example did.

    use warnings; use strict; my %foo = map { $_ => undef } qw( employees rates departments ); for ( keys %foo ) { open $foo{$_}, '>', "files/$_.txt" or die "Cannot write to $_.txt: $ +!\n"; } for ( keys %foo ) { print { $foo{$_} } "$_\n"; } for ( keys %foo ) { close ( $foo{$_} ) or die "Cannot close $_.txt: $!\n"; } for ( keys %foo ) { print { $foo{$_} } "$_\n"; }

    There's nothing special about filehandles after a fork. Closing the files through the filehandles in the parent does not close them in the child. Closing them in the child does not close them in the parent. Closing them in one child does not close them in other children. By keeping the files you open grouped in a convenient container data structure through which you can iterate, you can close them just as easily after a fork as in a single process.

    use warnings; use strict; my %foo = map { $_ => undef } qw( forked_1 forked_2 forked_3 ); for ( keys %foo ) { open $foo{$_}, '>', "files/$_.txt" or die "Cannot write to $_.txt: $ +!\n"; } print STDERR "I am $$, and I am the parent.\n"; if ( my $kidpid = fork ) { my $deadkid = wait; print "$$ says: $deadkid is no more. Now I will write to the files +.\n"; for ( keys %foo ) { print { $foo{$_} } "$$: $_\n"; } } elsif ( defined $kidpid ) { print STDERR "I am $$.\n"; for ( keys %foo ) { close ( $foo{$_} ) or die "Cannot close $_.txt: $!\n"; } for ( keys %foo ) { print { $foo{$_} } "$$: $_\n"; } } else { print STDERR "Failed to fork!?!?!?: $!\n"; }

    You may note that the parent waits until after the child has exited before it even attempts to write to the files. The child cannot write to them, because it closed its files through the handles. The parent has no problem, because the files are still open in the parent.

    Grouping filehandles in an array or hash is nothing revolutionary. It can make your life (well, your programming task, anyway) a whole lot easier if you need to treat data as a group to keep it grouped. Filehandles aren't just simple string or numeric data, but they are data. It's not just for forked processes, either. Another example application for this technique is in fact a non-forking server process that will accept an unknown number of connections. Anything that needs to open all the files in a directory and keep them all open at once rather than opening and closing them serially is a good candidate, too. Any program which opens a varying number of files, pipes, or sockets for any reason can benefit.

Re: how to close all files
by BrowserUk (Patriarch) on Oct 23, 2008 at 18:06 UTC
      Thanks to all, this last loop looks quite good for me.