Re: Re: Re: How to find a memory leak - appears to be "system" calls that are responsible

... if the system call to MailPush is omitted ... it runs like a charm.

Sounds like a problem with MailPush, then, not with gzip or mv. My guess is that MailPush is trying to be helpful by daemonizing itself to send the mail and returning immediately. Then it up and loads the whole file into memory before mailing it.

Try replacing MailPush with cat $splitoutfilename > /dev/null or something similar. If that works, try replacing it with Mail::Mailer or something similar. If that works, you're done! If it doesn't, use top to investigate while parsing a smaller file, one that doesn't completely hose the system.

Comment on Re: Re: Re: How to find a memory leak - appears to be "system" calls that are responsible Download Code

Replies are listed 'Best First'.
Re: Re: Re: Re: How to find a memory leak - appears to be "system" calls that are responsible by naum (Initiate) on May 15, 2004 at 15:09 UTC
>>Sounds like a problem with MailPush, then, not with gzip or mv. My guess is that MailPush is trying to be helpful by daemonizing itself to send the mail and returning immediately. Then it up and loads the whole file into memory before mailing it. No because I can string all those calls to MailPush in a ksh for loop and memory usage never goes above 8M whereas the Splitter just leaks memory out the wazoo when using "system" to invoke the process...	[reply]
Re: Re: Re: Re: Re: How to find a memory leak - appears to be "system" calls that are responsible by TilRMan (Friar) on May 15, 2004 at 16:11 UTC
... I can string all those calls to MailPush in a ksh for loop and memory usage never goes above 8M ... Well, then do that! I'm sorry, but I at least need more information. top! See how many processes are running at a time, and how much memory they each use. Since you are using system() everywhere, you should never have more than two processes: perl and whatever's in the `system()` (plus the extra shell that `system()` gives you). top combined with `perl -d` should let you find the exact point where the system bogs down. How about some of this `logit()` output? If logit() is timestamped, it could be useful. Include `$!` in your log messages after a `system()`; this will explain the `-1`s. Did the original script suffer from this problem? What are these files' names and how do they get called? Which is the library and which are the scripts?	[reply] [d/l] [select]