Re: Apache log piped to Perl

This is a totally wild guess, so please feel free to throw rocks at my head if I'm wrong.

Is it OK to open a file and then write to it from forked children and then close it from the parent? It seems to me that you'd get all kinds of horrible problems with multiple children trying to write at the same time... Maybe SDBM has a locking system and the parent process holds the lock? That would cause what you're seeing.

I think you need some kind of IPC to organize everyone so they don't step on eachother while writing to the data base file.

If I were doing this, first I'd seriously reconsider the value of forking off those children. If this were only one process without the forking it would be way easier. If I decided that I really needed that higher performance, I'd use threads with queues (Thread::Queue). I don't know what the equivalent to thread queues in a forking system is, but maybe that'd be easier.

Threads aren't really any harder than forking, in my experience.

See: perlthrtut, perlipc, Thread::Queue

Comment on Re: Apache log piped to Perl

Replies are listed 'Best First'.
Re^2: Apache log piped to Perl by Anonymous Monk on Aug 08, 2006 at 18:38 UTC
Is it OK to open a file and then write to it from forked children and then close it from the parent? That's a good question. My instinct is not to touch the hash tied by the parent from inside children in any way. I don't fork until the parent is done accessing %seen, which is ultimately untied from the parent as well. It seems logical to divide the labor by having the parent alone handle a very fast and reliable tied hash while forking off slower DNS queries to children who may add firewall rules based on the results. I'd seriously reconsider the value of forking off those children. Something like it has to be done or else a slow function in that while loop prevents Apache from serving requests. So far I'm seeing very few children in "top" on 1 second delay on a busy webserver, one or two at most, with no noticable slowdown.	[reply]

Replies are listed 'Best First'.

Re^2: Apache log piped to Perl
by Anonymous Monk on Aug 08, 2006 at 18:38 UTC

Is it OK to open a file and then write to it from forked children and then close it from the parent?

That's a good question. My instinct is not to touch the hash tied by the parent from inside children in any way. I don't fork until the parent is done accessing %seen, which is ultimately untied from the parent as well. It seems logical to divide the labor by having the parent alone handle a very fast and reliable tied hash while forking off slower DNS queries to children who may add firewall rules based on the results.

I'd seriously reconsider the value of forking off those children.

Something like it has to be done or else a slow function in that while loop prevents Apache from serving requests. So far I'm seeing very few children in "top" on 1 second delay on a busy webserver, one or two at most, with no noticable slowdown.

[reply]