#!/usr/local/bin/perl

use POSIX qw( sysconf _SC_OPEN_MAX );

$max = sysconf( &_SC_OPEN_MAX );
print "This process can have ",  $max, " open files\n";


# We all ready have STDIN, STDOUT, and STDERR open
$x = 3;

while( $x < $max ) {
   open( $x, "/etc/motd") ||
      die "cannot open file $!\n";
   $x++;
}

print "Should not have failed!\n";
print "Should fail now:\n";

open( $x, "/etc/motd" ) ||
  die "cannot open file $!\n";
[download]

-derby

A coworker ran into a problem where he was creating socket connections in an eval block with an alarm() to time them out if he didn't receive a response in a certain time period. It worked really really well, he was able to speed up his code by huge amounts. Until he inexplicably started running out of FD's.

It turned out that the FD's weren't being released after timing out of the eval block, so every time it timed out he accumulated another open FD that never went away (and they were being created implicitly inside perl, actually, so he didn't even have a handle to call close() on).

I'm not sure if this is at all related to your problem, but you might want to check on it. Most systems I've played with have the number of available FD's up in the thousands, so if you really only have a few hundred open, I'd think something weird is going on if you're getting errors.

Alan

UPDATE: *sigh* Pay no attention to the man behind the curtain

~~~ This is a curtain ~~~

Perl comes with a standard module for handling (UPDATE: the non-socket case of) this, it is FileCache. Perl 5.8 will have an improved version that is available over there (or in bleadperl of course, require perl 5.6+).

--
perl -pew "s/\b;([mnst])/'$1/g"

You can check from your shell (man ulimit or check your shell's manpage), or install BSD::Resource and use getrlimit.

$ perl -MBSD::Resource -le '@l = getrlimit(RLIMIT_NOFILE); print "soft
+ $l[0]\thard $l[1]"'
soft 1024        hard 1024
[download]

I can't keep a counter becouse my goal is make perl MODULE from this script (something like LWP::Parallel, but more efficient becouse of non-blocking sockets inplace of select()).
And this module will be used from user script... and this script can open his own files... any number of files... and I don't know in my module how many files open main script.


Update:I never say that I open only 200 sockets! To download 200 urls/sec I must open much more than 200 socket (~900), but after I open them and reach speed of 200 url/sec, I'm opening one new socket after one url downloaded and it socket is closed.
So, when script started it open ~900 sockets, and every second 200 from these 900 sockets will be closed and new 200 open.

powerman,

unfortunately a public api for that piece of the task structure has never been exposed. I looked for some api but couldn't find one. If you're opening pretty fast, you could always use fileno to let you know how close you are to the max but once you get going heavy and start closing, reuse will kick in and that will break.

I have to ask but what are you going to do when you get close to the max? Sleep? Can't you replicate that same behaviour by wrapping the open(s) in an eval? It seems to me, the following would be equivalent:

# the way you want
if( magical_how_many_open() < $what_i_need ) {
   do_wait_some_how();
}

# the way you're being pushed
eval {
   open( $fh, "whatever" );
};

if( $@ ) {
 if( $@ eq "Too many open files" ) {
  do_wait_some_how();
 } else {
  degrade_gracefully();
 }
}
[download]

-derby