I am merging files for an index, I worked around a hack to allow logarithmic merging without any heap structures just by clever tracking of numbers.
I'd have the in-memory index construction worked out, and the merging of 2 sub-indexes also works nicely, this is the last step in getting it to work, i'd like to have merging of the old indexes going on while new indexes are made. I can also try adding this code to the separate merging file, calling itself recursively or iteratively. | [reply] |
So, it looks like you're confident in the merging algorithm, so the real question is spawning the second process to merge the old indexes. Again, the pointers above from other monks are necessary reading if you're going to use fork() and exec(), but in general the following idiom is common:
use POSIX ":sys_wait_h";
my %kids = ();
# Signal Handler for SIGCHLD
sub sigchld_handler{
while (($pid = waitpid(-1, &WNOHANG)) > 0) {
delete $kids{$pid};
}
$SIG{CHLD} = \&sigchld_handler;
}
$SIG{CHLD} = \&sigchld_handler;
# You can repeat this for as many secondary
# processes as you need to merge the old indexes
for (1..3) {
if (my $pid = fork) {
# Parent Process keep track of forked children
$kids{$pid} = 1;
}
else {
# Child process
# ... do merging of old indexes here
sleep(3);
exit 0; # MUST EXIT HERE
}
}
while (keys %kids > 0) {
sleep(1); # wait for all children to finish
}
exit 0;
The SIGCHLD handler is there to let you know which of your child processing doing the merging completed successfully (or not), giving you a way to handle them properly. See the previously pointed to documentation about getting the actual exit value, signal and so forth from the completed child process. But, this framework should be enough to get you going if you have the merging algorithm down pat. | [reply]
[d/l] |