How do I fork in Win32 Perl?

jira0004 has asked for the wisdom of the Perl Monks concerning the following question:

Originally titled: fork in Perl on Windows 2000 -- I want to use fork to perform many file copies in parallel on Windows 2000

Hi,

I have a question regarding fork in Perl and Windows 2000.

First, let me explain what I am doing and why I want to use fork.

I am creating a "data image" whose source consists of many files possibly in many directories. I may need to copy these files from a network mapped drive onto a local drive to assure that my "data image" is constructed correctly.

File copy is of course an I/O operation. There is lots of inefficiency and I thought it would make sense to copy many files in parallel instead of copying each file one at a time.

copy itself is a synchronistic operation; your code blocks while you copy. It waits on the copy call until the copy completes and the call returns.

I want to launch many copies at the same time to achieve copy in parallel and reduce the transfer time by increasing the throughput.

So I plan on doing the following (pseudo-code):

foreach file in file_list {
    fork;

    if ( in child process ) {
        perform copy from file to destination;

        exit;
    }
}
[download]

I know that fork did not use to work in Perl on any Windows platform. I know that it now kind of works; fork results in successful fork on Windows, process not always stable after fork depending on operations to perform.

Here is what I want to know:

On UNIX, I'd perform all this forking, then if I wanted to wait until all of the child processes completed, I'd perform a waitpid until all of my child processes died, then I'd continue. On UNIX you really need to "reap" these child processes using waitpid or you produce "zombie" children which can take up operating system resources and cause problems for your process set (you can run out of table space for producing children).

Here finally is what I really want to know:

What do I do on Windows after doing all this forking??? I want to fork a bunch of child processes, have the child processes do file copy, have the parent process wait for the child processes to complete, reap the dead child processes so I don't screw up the system resources for my process set, and then continue executing the rest of my code after finishing this "parallel file copy". On UNIX I'd fork and then use waitpid until all of my child processes complete. What do I do on Windows??? Any help, input suggestions?

I've done quite a bit of system programming in Perl and C especially on POSIX (UNIX) compliant systems, but I've avoided too much system programming in Perl on Windows, and I've never done this sort of thing on Windows in Perl -- forkinig, doing stuff, waiting for child processes, continuing on. Thus, any advice, input or help would be useful, desired and appreciated.

Thanks for any helpful info regarding this topic.

P.S. I did a search on fork and window on the discussion groups for this site; there were a lot of questions regarding systems programming and fork, but I didn't find any helpful postings about making this work on Windows 2000.

--- Peter Jirak

Email: jira0004@yahoo.com

Edited: ~Fri Jun 28 15:03:33 2002 (GMT), by footpad:
Shortened title, per Consideration

Comment on How do I fork in Win32 Perl? Download Code

Replies are listed 'Best First'.
Re: How do I fork in Win32 Perl? by Rex(Wrecks) (Curate) on Jun 26, 2002 at 19:53 UTC
Fork is not stable in windows yet, fork() does not actually launch a process in Win32 Environment, it uses threads instead of process and since threading is still experimental in Perl...you get the picture. Now that we have done the "Why not to use fork() in Win32", I would suggest looking into Win32::Process. I have successfully used it to spawn and manage process in Win32. Also if you can find Win32::IProcess, it works very well too. The problem with both of these is that your spawned process typically has to be an externel app, which means your script has another dependancy. AFAIR Your waitpid methods should work just fine in Win32 should you decide to continue using fork(), even though you are working on threads, Perl does the magic underneath using ThreadID instead of PID. Believe me, I am so waiting for threading to be stable in Perl, looks like 5.8 might be the ticket, but my guess is we will be waiting for Perl 6. "Nothing is sure but death and taxes" I say combine the two and its death to all taxes! Edited: ~Fri Jun 28 15:06:13 2002 (GMT), by footpad: Changed title to reflect parent and tweaked HTML Formatting for XML compatibility	[reply]

Replies are listed 'Best First'.

Re: How do I fork in Win32 Perl?
by Rex(Wrecks) (Curate) on Jun 26, 2002 at 19:53 UTC

Fork is not stable in windows yet, fork() does not actually launch a process in Win32 Environment, it uses threads instead of process and since threading is still experimental in Perl...you get the picture.

Now that we have done the "Why not to use fork() in Win32", I would suggest looking into Win32::Process. I have successfully used it to spawn and manage process in Win32. Also if you can find Win32::IProcess, it works very well too.

The problem with both of these is that your spawned process typically has to be an externel app, which means your script has another dependancy.

AFAIR Your waitpid methods should work just fine in Win32 should you decide to continue using fork(), even though you are working on threads, Perl does the magic underneath using ThreadID instead of PID.

Believe me, I am so waiting for threading to be stable in Perl, looks like 5.8 might be the ticket, but my guess is we will be waiting for Perl 6.

"Nothing is sure but death and taxes" I say combine the two and its death to all taxes!

Edited: ~Fri Jun 28 15:06:13 2002 (GMT), by footpad:
Changed title to reflect parent and tweaked HTML Formatting for XML compatibility

[reply]