at the office I am working on threaded perl script on a linux system. at first just as a test I had the script work on a sub set of the large data file.

I gradually increased the amount of data to be processed by the perl script and everything seemed to be fine. I seem to have reached a limit of some kind, my script now dies with a killed message and the exit status is 137 (which means killed by a SIGKILL signal)

I have the main program create and load a threaded queue with all of the data records from the large data file. I next have the main script start all the threads (currently 35 threads). First the thread sets an element in a shared hash to indicate that it has started. Next the thread loops over elements from the queue using a non-blocking fetch until nothing is returned. Finally, the thread sets the shared hash to indicate it is done.

My script detaches the threads after they are started.

After all of the threads are started, the main script goes into a loop waiting for all the threads to mark their status flags as done. After a certain period of time if not all the flagfs indicate DONE then the main script prints a message and exits.

Both the main script and the threads use signal handling. No signals were caught.

I am guessing that my script was likely killed due to a too much memory used problem. Does a perl threaded queue store its elements in memory ?

Any suggestions on what to check for memory issues or any other likely suspects ?

In reply to need help debugging perl script killed by SIGKILL by expo1967

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.