Beefy Boxes and Bandwidth Generously Provided by pair Networks
Do you know where your variables are?
 
PerlMonks  

Program hangs

by Stegalex (Chaplain)
on Jan 27, 2002 at 21:39 UTC ( [id://141947]=perlquestion: print w/replies, xml ) Need Help??

Stegalex has asked for the wisdom of the Perl Monks concerning the following question:

I have a daily Perl script that runs and sends out several hundred emails (opt-in only of course). For the last several days, I have been noticing that it is not completing. It just "hangs" and stays on the process table. There is nothing printed either in the logs, in stderr, or in stdout. When I modified the program to print lots of debugging information, the output just stops halfway through printing a variable (it doesn't even complete printing the variable). My first thought was that I had run out of disk space, but running df showed that space was not a problem. Perhaps memory is a problem, although in the past, I have seen Perl print "out of memory!" to stderr and the program dies (not the program in question). This program is NOT dying, it just hangs out. It's not just moving slowly either and system performance does not seem to be impacted. I am sure that I am not trapped in an infinite loop because I have so many print statements that if I were in a loop, the logs would eventually consume all remaining disk space.

So, the question I have is this: If I keep the process running, is there a way to look inside to find out what's going on?

While I would love to post the code, company policy prohibits.

Thanks!

P.S. Perl 5.6.0
R.H. Linux
Oracle 8i

Replies are listed 'Best First'.
Re (tilly) 1: Program hangs
by tilly (Archbishop) on Jan 27, 2002 at 23:22 UTC
    If your output stops halfway through printing a variable, the odds are that your debugging output is Suffering From Buffering. Fixing that won't fix your original problem, but may help you debug it.

    As for what it is, many possibilities exist. You might be just triggering sendmail's check for being used to send spam (at which point it deliberately slows down). Perhaps you have one particular place you send information which hangs for reasons beyond your control. (In which case it might make sense to use Parallel::ForkManager to run several processes in parallel, so that one hangs and the rest go to their destination.) It could be something else.

    For a random example of "something else", I am in the process of diagnosing a sporadic hang at work which I have tracked down to probably being delays in closing a database connection. (It is now in the hands of some network folks.)

      Hmmm. I am pretty sure it's not an SMTP problem as the program does not directly interact with SMTP, rather, it sends a command to another server which then forms and sends a mail message. Also, I am using qmail instead of sendmail. Thanks for your advice. I will look into your other tips.
Re: Program hangs
by Fletch (Bishop) on Jan 27, 2002 at 22:07 UTC

    A tool like strace that can dump out what system calls are being made can provide insight into strange hangs like this. You may want to have the services of a seasoned C programmer or sysadmin handy if you're not familiar with it.

Re: Program hangs
by beebware (Pilgrim) on Jan 27, 2002 at 22:30 UTC
    Trying running the code with perl -wc programname.pl as this has caught many hangs that I've experienced (sometimes just a closing multi-line quote missing can hang the entire thing).
    Use use strict; as well to double-check everything, and if need be add debug lines (printing to the console - not to a file: you'll have less to go wrong if it is displayed rather than written) after every statement.
    Oh, since you're using Oracle (I'm guessing via DBI), ensure that all your modules are up-to-date and turn on debugging on the DBI module as that will help as well.
Re: Program hangs
by n3dst4 (Scribe) on Jan 27, 2002 at 22:28 UTC
    If it's single-threaded and you're sending emails, maybe an SMTP connection is getting hung. If neither you nor the far end server times out, it would appear that your program had stopped.

    Alternatively, you might have several slow connections, and when you come to look at the progress, it just happens to be stuck on one of them. How long have you left it just sitting there? It might come back after a while.

      It is single-threaded, although it doesn't interact directly with SMTP, it passes a command to a different machine. The other machine actually forms the mail message.

      The program is stopping printing right in the middle of printing a variable. This is the sort of thing that happens when you run out of disk space, although I have plenty of space in the filesystem.

      The process can hang for days without moving. It took me a couple of days before I noticed the problem because there were no obvious signs of errors such as stderr, stdout, logfile entries.
        What does the other machine (the receiver) think is happening? Are the commands being passed down a constantly-open channel which is getting clobbered by an intervening firewall after a certain length of time?

        Oh, and if this is on Linux, cat /proc/sys/fs/file-nr while the program is hung. You may have run out of filehandles (although you'll have a hard time launching a shell etc. if that's the case).

Re: Program hangs
by xylus (Pilgrim) on Jan 27, 2002 at 21:47 UTC
    While I'm sure most of us here would love to help, but without a more specific idea of what is going on, there isn't much help to offer except to point you to some resources such as: perldebug.

    However, if you could be more specific (perhaps psuedo-code) we could offer a little more insight.

    -xylus
Re: Program hangs
by atcroft (Abbot) on Jan 27, 2002 at 23:29 UTC

    Thoughts off the top of my head:

    • time-outs (mentioned above)
    • memory/space issues (mentioned above)
    • program code issues(mentioned above
    • unexpected data

    The first three were mentioned above, and in better detail than I could do.

    What do I mean by that last one? The possibility of invalid data (invalid character in or bad/unexpected email address/name/whatever). You might consider wrapping your sending code/call in an eval() statement then examining the values of the $!, $@, $?, and/or other variables (as appropriate to your code) and logging any content they have (along with the data you used on that pass).

    I hope that helps, and hope what monks wiser than I will chime in, so we both may learn.

      Good thought, but the data is clean. That's one of the first things I checked.
A reply falls below the community's threshold of quality. You may see it by logging in.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://141947]
Approved by root
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others romping around the Monastery: (8)
As of 2024-03-29 15:16 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found