Re^2: Lots of memory usage with fork&copy

I'm copying different files on different locations, some are over internet (on windows shares) and sometimes the copy process takes some time or the connections fails.
Because of this lags I've got to do things in parallel. I have to copy after another process does its job so I have to have some kind of synchronisation.
I could also execute another .pl everytime but I don't think that that is very efficient.

I watched the number of threads and it doesn't grow more than 2, so it's ok. Probably it doesn't free the memory on exit&waitpid. Any ideas where exactly to report this as a bug? Directly to activestate's site or elsewhere?

Thanks for your replies,
andrei

Comment on Re^2: Lots of memory usage with fork&copy

Replies are listed 'Best First'.
Re^3: Lots of memory usage with fork&copy by samtregar (Abbot) on Apr 25, 2007 at 17:43 UTC
I'm copying different files on different locations, some are over internet (on windows shares) Red flag on the play! You're copying files over the internet using Windows shares? That's wrong in a so many ways - performance, security, sanity... Stop right now and pick a new tool - rsync, scp, HTTP and FTP all spring to mind. All have solid Windows implementations which should be fairly easy to get running concurrently. -sam	[reply]
Re^4: Lots of memory usage with fork&copy by andreidf (Novice) on Apr 27, 2007 at 08:04 UTC
Security.. not really. It's a VPN connection. Performance? Yes. Sanity? I have to do this under this circumstances. I don't control the firewall so I have to use a windows share. I'll try to port the tools to use ftp in the near by future but till then I still think perl does something wrong and leaks memory. Please give me hints how to stop the memory leak and don't tell me that rsync is better because I know that. :) I found it a little bit strange that just by using win32 module ("use win32") perls leaks less memory by fork,copy,exit. Can you give me a direction on how to investigate this? I am new to perl and I have created just small scripts, nothing complicated or advanced Best regards, andrei	[reply]
Re^5: Lots of memory usage with fork&copy by samtregar (Abbot) on Apr 27, 2007 at 18:00 UTC
Well, I'm really not a Windows programmer. When I have to do something on Windows I try to keep it as simple as possible. One thing you could try is actually spawning a new process rather than using Perl's thread-based fork(). I believe you do that with Win32::Process. You could spawn a separate process to copy each file, which might help keep memory use under control. -sam	[reply]