The problem is that there are too many files open. Without knowing the exact message and what messages are in your system I can't tell you whether the limit that is being exceeded is the number your process may have open, or the whole system. At a guess if it doesn't say anything about the system then it is your process. But to get a better sense you should look through the possible system messages to figure out which error message you're getting. This line of code will give you those messages: 
perl -le 'print ($! = ++$_) until $! = ~/^Unknown/'

[download]
On my system two of the messages are: 
Too many open files in system
Too many open files

[download]
The first message would indicate a global problem on my machine, the second one that is specific to my process.

If it is global, you need to look at what is happening on the machine as a whole. It may not be easily fixable (though in Linux you can get them to up the limits, as others have told you.) If it is local to your process then the change in behaviour is driven by data that you have. Somehow you're opening files and not closing them. Without looking at your code I can't tell you where that is happening. However what you need to do is close some of your open files.

You can do this explicitly, or by using the built-in FileCache to close and reopen filehandles behind the scenes. Do note that it has a bug where it does not reset the position of filehandles if it opens and closes them. If that matters to you then I would create a tied class that can open and close files like FileCache does but also keeps track of positions.

If the problem is global it might still be the fault of your process. Aggressively avoiding multiple files open is not a bad idea. But the odds are that you aren't the culprit of the system problem. Instead it has to do with system load and, somewhere, some process is using lots more filehandles than it needs.

In reply to Re: How to debug "Too Many Open Files" Error by tilly
in thread How to debug "Too Many Open Files" Error by tonyb48

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.