Re: Net::Server::PreForkSimple and Children

Funny you should ask. This is one of the more common questions fielded for Net::Server.

In either of PreFork or PreForkSimple you are going to have this problem. As salvix already pointed out, forked processes get a snapshot of the state of the parent at the time of fork. Any updates to the parent after that point will not be reflected in the already forked processes. You need a method to share the data.

For caching the data I'd recommend one of Cache::Memcached or Cache::FastMmap. Both of these are extremely fast. If all you are doing with the server is serving the cached data then Cache::Memcached already does everything you need (if you are doing more and just want a way to share common data then you still will need the Net::Server).

In the end I'd benchmark a few of the Cache systems (Memcached, FastMmap, and IPC::Shareable) and see which one performs the best under your load circumstances.

Another solution that is even faster - assuming you don't care about whether the data is shared between the processes (if you don't care that each process has a full copy of the data in memory) - you could go with a hybrid approach where the parent updates the Cache and the child processes only periodically check the cache for updates and then copy the entire cache into perl variables local to the child processes. For 99% of the server needs this would probably be overkill.

I think it is about time to add a section discussing this to the PreFork perldoc.

my @a=qw(random brilliant braindead); print $a[rand(@a)];

Comment on Re: Net::Server::PreForkSimple and Children

Replies are listed 'Best First'.
Re^2: Net::Server::PreForkSimple and Children by Rhandom (Curate) on Nov 30, 2005 at 17:49 UTC
So that was the right answer for the wrong question. You are doing what I suggested in the paragraph beginning with "Another solution" which is that each child occasionally loads its own copy of the data - periodically. Your implementation appears sound. I think it would be helpful to also log the pid with each of your log messages so we could get a better idea which ones are reloading - and if the same one is reloading. It is entirely possible that some of your preforked processes haven't been touched yet - and when they finally are then you get the old time stamp you have shown. my @a=qw(random brilliant braindead); print $a[rand(@a)];	[reply]
Re^3: Net::Server::PreForkSimple and Children by hubb0r (Pilgrim) on Nov 30, 2005 at 20:38 UTC
Thanks for your replies... and yes it would be nice to have a bit more documentation regarding this type of scenario in the Prefork or PreforkSimple docs... The entire Net::Server module is very different than what I'm used to using in the way that I had to modularize my program and have it call itself during execution. So, I actually did go and turn on pid logging after posting this message, and have been watching what's been happening at this point. I've also changed the config file to spawn only one server at a time so I can attempt to see exactly what is happening: `min_servers 1 max_servers 1 max_requests 1000 log_file /var/log/speak_db_names.log log_level 3 pid_file /tmp/speak_db_names.pid background 1 port 3307` [download] Here is a current snippet of the logfile: 16265 - Reusing DB Cache 0 Seconds old 2005/11/30-15:30:00 CONNECT TCP Peer: "172.17.10.67:50661" Local: "172 +.17.10.75:3307" 16265 - Reusing DB Cache 0 Seconds old 2005/11/30-15:30:01 CONNECT TCP Peer: "172.17.10.67:50668" Local: "172 +.17.10.75:3307" 16265 - Reusing DB Cache 1 Seconds old 2005/11/30-15:30:02 CONNECT TCP Peer: "172.17.10.67:50673" Local: "172 +.17.10.75:3307" 16265 - Reusing DB Cache 2 Seconds old 2005/11/30-15:30:03 CONNECT TCP Peer: "172.17.10.57:35875" Local: "172 +.17.10.75:3307" 16265 - Reusing DB Cache 3 Seconds old Starting "1" children 2005/11/30-15:30:06 CONNECT TCP Peer: "172.17.10.67:50683" Local: "172 +.17.10.75:3307" 16269 - DB Cache expired: 135784 Seconds Old... Reloading 16269 - Now: 1133382606 Last Upd: 1133246822 2005/11/30-15:30:06 CONNECT TCP Peer: "172.17.10.67:50687" Local: "172 +.17.10.75:3307" 16269 - Reusing DB Cache 0 Seconds old Starting "1" children 2005/11/30-15:30:09 CONNECT TCP Peer: "172.17.10.57:35883" Local: "172 +.17.10.75:3307" 16273 - DB Cache expired: 135787 Seconds Old... Reloading 16273 - Now: 1133382609 Last Upd: 1133246822 2005/11/30-15:30:10 CONNECT TCP Peer: "172.17.10.57:35885" Local: "172 +.17.10.75:3307" 16273 - Reusing DB Cache 1 Seconds old 2005/11/30-15:30:10 CONNECT TCP Peer: "172.17.10.57:35887" Local: "172 +.17.10.75:3307" 16273 - Reusing DB Cache 1 Seconds old 2005/11/30-15:30:12 CONNECT TCP Peer: "172.17.10.57:35889" Local: "172 +.17.10.75:3307" 16273 - Reusing DB Cache 3 Seconds old 2005/11/30-15:30:13 CONNECT TCP Peer: "172.17.10.57:35891" Local: "172 +.17.10.75:3307" 16273 - Reusing DB Cache 4 Seconds old 2005/11/30-15:30:14 CONNECT TCP Peer: "172.17.10.67:50706" Local: "172 +.17.10.75:3307" 16273 - Reusing DB Cache 5 Seconds old 2005/11/30-15:30:15 CONNECT TCP Peer: "172.17.10.57:35893" Local: "172 +.17.10.75:3307" 16273 - Reusing DB Cache 6 Seconds old 2005/11/30-15:30:15 CONNECT TCP Peer: "172.17.10.57:35895" Local: "172 +.17.10.75:3307" 16273 - Reusing DB Cache 6 Seconds old Starting "1" children 2005/11/30-15:30:33 CONNECT TCP Peer: "172.17.10.57:35897" Local: "172 +.17.10.75:3307" 16276 - DB Cache expired: 135811 Seconds Old... Reloading 16276 - Now: 1133382633 Last Upd: 1133246822 Starting "1" children 2005/11/30-15:30:58 CONNECT TCP Peer: "172.17.10.57:35899" Local: "172 +.17.10.75:3307" 16279 - DB Cache expired: 135836 Seconds Old... Reloading 16279 - Now: 1133382658 Last Upd: 1133246822 Starting "1" children 2005/11/30-15:31:25 CONNECT TCP Peer: "172.17.10.57:35901" Local: "172 +.17.10.75:3307" 16282 - DB Cache expired: 135863 Seconds Old... Reloading 16282 - Now: 1133382685 Last Upd: 1133246822 2005/11/30-15:31:26 CONNECT TCP Peer: "172.17.10.57:35902" Local: "172 +.17.10.75:3307" 16282 - Reusing DB Cache 0 Seconds old Starting "1" children 2005/11/30-15:31:29 CONNECT TCP Peer: "172.17.10.57:35904" Local: "172 +.17.10.75:3307" 16285 - DB Cache expired: 135867 Seconds Old... Reloading 16285 - Now: 1133382689 Last Upd: 1133246822 2005/11/30-15:31:29 CONNECT TCP Peer: "172.17.10.57:35905" Local: "172 +.17.10.75:3307" 16285 - Reusing DB Cache 0 Seconds old Starting "1" children 2005/11/30-15:31:37 CONNECT TCP Peer: "172.17.10.57:35907" Local: "172 +.17.10.75:3307" 16288 - DB Cache expired: 135875 Seconds Old... Reloading 16288 - Now: 1133382697 Last Upd: 1133246822 2005/11/30-15:31:37 CONNECT TCP Peer: "172.17.10.57:35909" Local: "172 +.17.10.75:3307" 16288 - Reusing DB Cache 0 Seconds old 2005/11/30-15:31:37 CONNECT TCP Peer: "172.17.10.57:35911" Local: "172 +.17.10.75:3307" 16288 - Reusing DB Cache 0 Seconds old 2005/11/30-15:31:38 CONNECT TCP Peer: "172.17.10.57:35913" Local: "172 +.17.10.75:3307" 16288 - Reusing DB Cache 1 Seconds old Starting "1" children [download] So I'm still confused. My config file says: spawn one server (min/max) to deal with requests, and serve 1000 requests per server, right? So if you look at the log file, you will see that sometimes on process is used multiple times, but then a child will be spawned, and seemingly the next x number of calls call different child processes, none of which have the updated cache information. I don't mind a single process getting the cache information when it expires, but I don't want them getting it every call, and some of them seem to do so. Maybe you can see what I'm doing wrong, if anything at all, or maybe explain a bit more to me about what is happening. I've only done a few other programs that fork, and none of them have been a client/server type, so this is a bit new to me, although I think I understand much of the concept. Thanks	[reply] [d/l] [select]