Re^2: Perl and 7zip and backticks

Hi Browser. Thanks for your reply, interesting if 7zip updates its progress via STDERR. I tried your example and 7zip seemed to update more often but in blocks of output not each line as it was being zipped (talking large GB) files here so it's easy to see it update per item. I'll fiddle around with your example more. Cheers

Comment on Re^2: Perl and 7zip and backticks

Replies are listed 'Best First'.
Re^3: Perl and 7zip and backticks by BrowserUk (Patriarch) on Feb 27, 2016 at 23:13 UTC
but in blocks of output not each line as it was being zipped Unfortunately that is par for the course with pipes; they have their own buffer. Indeed, that pretty much defines what a pipe is: a piece of shared memory buffer space. The pipe will accumulate output from the writer until the buffer (often 4K) fills, and only then does it start to satisfy the reader. In most cases that is a good thing as it prevents the processes giving up time-slices and switching back and forth in lockstep. Other than setting up the pipe yourself -- a messy procedure easy to get wrong -- and using the barely documented ioctl calls to specify the buffer size(s), I don't know any way of avoiding that. With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday' Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error. "Science is about questioning the status quo. Questioning authority". I knew I was on the right track :) In the absence of evidence, opinion is indistinguishable from prejudice.	[reply]

Replies are listed 'Best First'.

Re^3: Perl and 7zip and backticks
by BrowserUk (Patriarch) on Feb 27, 2016 at 23:13 UTC

but in blocks of output not each line as it was being zipped

Unfortunately that is par for the course with pipes; they have their own buffer. Indeed, that pretty much defines what a pipe is: a piece of shared memory buffer space.

The pipe will accumulate output from the writer until the buffer (often 4K) fills, and only then does it start to satisfy the reader. In most cases that is a good thing as it prevents the processes giving up time-slices and switching back and forth in lockstep.

Other than setting up the pipe yourself -- a messy procedure easy to get wrong -- and using the barely documented ioctl calls to specify the buffer size(s), I don't know any way of avoiding that.

With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.

"Science is about questioning the status quo. Questioning authority". I knew I was on the right track :)

In the absence of evidence, opinion is indistinguishable from prejudice.

[reply]