Increase IO buffer

antivirus64 has asked for the wisdom of the Perl Monks concerning the following question:

Hello everyone,

I'm implementing IPC API via named pipes. I've never done this before. I would like to skip the hassle of prepending the length of each messsage in the stream, so the other end knows how much bytes to read for each of them. My approach is to close the pipe every time a message ends, and then reopen it immediately before data needs to be transmitted again. My simple tests (with the help of cat), suggest that it is going to work as I want. However, I want to make sure that Perl is going to write to the pipe only after close $pipe. I read somewhere that the internal buffer is 8kb. How can I increase it? If it's possible to tell Perl not to write unless explicit close is invoked would be even better.

Thanks

Comment on Increase IO buffer Select or Download Code

Replies are listed 'Best First'.
Re: Increase IO buffer by Corion (Patriarch) on May 31, 2017 at 19:07 UTC
I'm not sure if I understand your setup correctly. Maybe you can show us a small ASCII-art image telling us which program is the server/parent and which is the client/child and how they are supposed to interact? From your description I think that your child is written in Perl and you want to connect to the child via a named pipe, write to it, and then keep on reading from the named pipe the response, and hope that you will not block writing. I think you should basically be able to achieve that by not writing anything from the child until it has read all the input and the input stream returns eof. Barring that, have you considered having your child process simply writing its response to a file? That way, you can seek within the file to your hearts content. Just `flush` the filehandle from the child after you've written all the data. There is a reason why one usually prepends the length of the expected input to the requests :)	[reply] [d/l]
Re: Increase IO buffer by morgon (Priest) on Jun 01, 2017 at 01:48 UTC
Evidently this depends on the system used. On Linux "man 7 pipe" gives you more information and indicates that the buffer capacity can be changed with an fcntl-call. hth	[reply]
Re: Increase IO buffer by salva (Canon) on Jun 01, 2017 at 07:23 UTC
Use something similar to chunked transfer encoding which would allow you to detect the end-of-message easyly (it is marked by a 0-length chunk).	[reply]
Re: Increase IO buffer by kcott (Archbishop) on Jun 01, 2017 at 08:16 UTC
G'day antivirus64, Welcome to the Monastery. Here's some links to information you might find useful. The first three are posts I made a few years ago. "Re: Question about communicating with a running perl script": Two separate scripts communicating via named pipes. "Re: Filehandle reference lost between parent and child": One script with two processes (via `fork`) communicating via named pipes. "Re^3: Filehandle reference lost between parent and child": A follow-up response to the post above. This discusses closing readers and writers and how that affects the buffer. "perlipc - Perl interprocess communication" has a whole section on "Named Pipes". The POSIX documentation has information on the `mkfifo()` function. — Ken	[reply] [d/l] [select]