wulfric has asked for the wisdom of the Perl Monks concerning the following question:

I have a perl program that runs through some thousands of files (emails) searching for certain characteristics. It runs properly but when the number of files exceeds some thousands, it end with "Terminated" and no other hint as to why. Runs for less than a minute before this happens. What's the likely cause? Out of memory?

It uses Email::MIME and a couple other cpan modules which work fine and were very helpful. Program is running on a remote linux server. I'm puzzled as to where to look for the cure? Thanks in advance.

  • Comment on Why is my perl program 'Terminated' without explanation?

Replies are listed 'Best First'.
Re: Why is my perl program 'Terminated' without explanation?
by john_oshea (Priest) on Mar 12, 2006 at 19:59 UTC

    If you have access to the logs on the remote machine, I'd suggest looking there first - if your program has exceeded any system-imposed limits, you may get an indication as to why. If you're not already, I'd suggest building in some sort of logging mechanism to your own code - this can be as basic as sprinkling print statements in the appropriate places.

    Given that you mention 'thousands' of files which are not likely to be enormous (if they're typical of common emails), I'd guess you were running into open file descriptor limits. Better logging will reveal all though.

    Hope that helps.

    2006-03-13 Retitled by planetscape, as per Monastery guidelines
    Original title: 'Re: Terminated'

[reply]
[d/l]

      Thanks for the reminder to look at the logs. The answer was there: "Out of Memory: Killed process 19956 (pr_email.pl)."

      Now to figure out why, but I suspect the objects created by Email::MIME are not being detroyed properly. Will have to look into that. Email::MIME itsself doesn't have a destroy method.

      2006-03-13 Retitled by planetscape, as per Monastery guidelines
      Original title: 'Re^2: Terminated'

[reply]
Re: Why is my perl program 'Terminated' without explanation?
by gloryhack (Deacon) on Mar 12, 2006 at 19:57 UTC
    The only time I've seen something like this, it was on a web hosting machine upon which there was a daemon watching resource utilization (CPU, memory, and time) and killing any processes that exceeded some predetermined thresholds. In that case, the fix was to abandon that web hosting provider.

    2006-03-13 Retitled by planetscape, as per Monastery guidelines
    Original title: 'Re: Terminated'

[reply]
Re: Why is my perl program 'Terminated' without explanation?
by Marza (Vicar) on Mar 12, 2006 at 19:54 UTC

    Sounds like you may have hit a limit. So it could be memory so what's the memory on the server? Did you run it through a debugger?

    2006-03-13 Retitled by planetscape, as per Monastery guidelines
    Original title: 'Re: Terminated'

[reply]
Re: Why is my perl program 'Terminated' without explanation?
by hesco (Deacon) on Mar 13, 2006 at 18:54 UTC
    The logs are your friend. If I'm working in a hosted environment, I ask the administrator to create an application specific system user and group, application specific access and error logs (if its a web app) and to assign my own personal user to the application's group, and use chown www-data::appgroup (assuming web app) to make the log file accessible to me, so I don't need to bother the administrator on this more than once. That permits me to monitor the logs myself. Another useful tool I recently discovered is Log::Log4perl. Makes implementing logging trivially easy. Of course it sounds like your script is run from a cli, making logging permissions easier. You can just create the logs you need in your own home directory.

    -- Hugh

[reply]

Back to Seekers of Perl Wisdom