And therefore the pattern match will match: <Dumped-data-from-%INFO>:DONE::MSG:<Dumped-data-from-another-%INFO>

That's definitely something I would recommend you look into. Another possibility is that $collected is something like ":MSG:stuff:DONE::MSG:stuff" when you do $collected = "", breaking the stream, or that :MSG: and :DONE: are a part of the Dumper output. Although I still don't recommend using Data::Dumper and especially eval for this, here are some steps you can take. First, make your start and end markers a little more robust: I think for example Data::Dumper wouldn't output a "\0" when its Useqq option is turned on (and you can even validate this by checking the string before you send it). Second, use a s/// regex to remove output you get from $collected instead of completely clearing it, and/or validate the format of $collected before you do.

If you still have trouble, it would be great if you could combine the tidbits of code you've provided into an SSCCE that reproduces the issue, then I'm sure we can get it debugged pretty quick.

Update: A few minor additions to the above.

Thank you for your comments.
I have created a small test-case which at least causes the same error-message to be generated.
(I appreciate that many causes may generate the same error-message).

In my small test-case, if I have a "corrupted" $dumped that is passed to eval, then I get the same error-message.
And by corrupted, I mean:
• that the collection/appending of snippets of string from read() onto $collected
• followed by the extraction of one framed message
• followed by the complete nulling of $collected without testing to see if there is a residue of text that should be preserved
could result in a ill-formated string that gets passed to eval.

And, in such an instance, I note that the result of the eval results in a %info that is empty.

(It never crossed my mind to consider that %info = %{ eval $dumper}</code} would result in a null %info.)
So this then becomes another possible trigger for break-pointing or printing out extra information at/around the failure point.

Thank you for your thoughts and time.
Andrew

I checked back in after a couple of days to see how you are making out?

In my suggestions at this node, I am assuming that the autoflush would have the same effect as manually flushing after your child's print statement. Further assumed is that this file event notification does work and means that the pipe is ready for reading. If the pipe has any data in it, then all of the data for a single msg should be coming forthwith and certainly fast enough to have no impact on GUI performance.

If both of those are true, then my suggested blocking read until DONE seen should work well and you don't need your read() statement. There should never be any "garbage" before the start of the frame, e.g. "MSG". Stopping reading after seeing DONE should leave the input pipe completely empty. Things are more complicated if 2 msgs could be "back to back". But since they are only sent once per 10 seconds, I see no problem. Of concern with "back to back" messages would be timing issues with "re-enabling" the file even notification mechanism - not sure how that would work.

A good assumption for these pipes is that they are lossless. Probability of corruption is on the order of a memory RAM error (which nowadays is very low).

I was wondering if there was some scenario where %INFO could actually be an empty hash. Thanks for clarifying that you've thought about that. Also, the possible 10K message is significant.

In your child, the autoflush part looks good to me. However, I am not sure exactly what will happen with more than one send buffer's worth of data. Probably the buffer size is 4K as a good guess. The "print" will not be "atomic". Could be separate 4K, 4K, then 2K writes.

In your parent, $num = read($child_fh, $buf, 1e6); you are assuming that all of the data that the child sent is available at once. Usually that will be true (what the child does is very fast), but sometimes that may not be true. What happens if you read the pipe between the sending of one of child's buffers? You would get everything sent so far, but perhaps not all that will be sent by the child. If the child has only put a single 4K packet into the pipe and is still working on the next one, your statement will just ignore the data that is to come.

Your parent read loop should read data until is sees ":DONE:". It probably would be ok to use a blocking call for this on a line by line basis. The reasoning is that the child's transmission job is fast. If a buffer containing the lines received don't start with ":MSG:", then you have a malformed message. I see no reason for state $collected; I would just use a normal "my" variable with appropriate scope.

You actually have a fair amount of time that can be spent in the parent read routine. You are reading this pipe because you got a notification that at least one byte was available. The child will be sending all of the other bytes forthwith. Make sure that you are getting the entire message. The GUI user will not notice a couple of ms of delay.

Update: If I understand correctly, you are getting about 8,640 messages per day (once every 10 seconds). One out of 20-30K messages fails? This one time per 25K sounds like an inter-process com failure. I would put some code in to detect "malformed message" (got DONE, but buffer doesn't start with MSG).

Update: I was thinking about this some more and if you were randomly polling the client, this kind of thing wouldn't happen nearly as often as once per 20K times - you are seeing a problem "relatively often". However, you have special "push" situation that increase the probability of an error. The parent will be notified very soon after a large data buffer is starting to be written to the pipe - I would think at least after the first 4k. The parent could get to the pipe before the client finishes sending. You could wind up with a partial message or a message containing fragments from a previously not completely received message.

I very much like the idea of what you did with adding :MSG: and :DONE: The parent should wind up synchronized so that it only gets complete messages, one at a time. After you see a :DONE: line, I don't think that there is any tricky stuff that you need to do in case there is another message "stacked behind" that one. The client only sends these things once every 10 seconds. When you see the :DONE: line, if the buffer doesn't have :MSG: right at the start of it, then you have a framing error. If there are no framing errors, then we into a different kind of mystery. Once established, you can assume that a connection like you have is without transmission errors.

Change the parent read loop to look for ":DONE" do not assume that a single read will do that.

To be more specific, (untested of course), in the child:
put beginning and ending key words on separate lines
change  print $parent_fh ":MSG:${dumped}:DONE:\n";
to print $parent_fh ":MSG:\n${dumped}\n:DONE:\n";

In the parent - something like this...:

# OK to use blocking calls, the data will be there
# don't read past the "DONE", wait for next file event
# otherwise this will "hang". 

my $msg;
while (my $line = <$child_fh>)
{
   $msg =. $line;
   last if $line =~ /^:DONE:\s*$/;
}

if ($msg !~ /^:MSG:/) {framing error!}
do what you need to do...
$msg = undef;
[download]