Perl and 7zip and backticks

Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

Replies are listed 'Best First'.
Re: Perl and 7zip and backticks by GrandFather (Saint) on Feb 26, 2016 at 02:10 UTC
There are no trivial fixes to the problem. Back ticks doesn't return the output from the process it runs until the process terminates. It may be that something like Archive::Zip is useful to you instead of calling out to the OS. Premature optimization is the root of all job security	[reply]
Re^2: Perl and 7zip and backticks by Anonymous Monk on Feb 26, 2016 at 03:17 UTC
Thanks for responding, sounds like I'll just have to live with it...	[reply]
Re^3: Perl and 7zip and backticks by Anonymous Monk on Feb 26, 2016 at 03:23 UTC
There are other modules which claim support for 7zip format, they may or may not provide ability to output a progress bar, and they may not be trivial to install :)	[reply]
Re: Perl and 7zip and backticks by BrowserUk (Patriarch) on Feb 26, 2016 at 04:12 UTC
The status information comes out via stderr, so you'll need to redirect that in order to capture it. Try: `open PIPE, '-\|', '"\Program Files\7-Zip\7z.exe" a -r -mx0 -tzip e:\the +Zip.zip 2>&1' or die $@;; while( <PIPE> ) { # do something with the status output here }` [download] Update: Here's the test I ran of my suggestion: Perl> open PIPE, '-\|', '"\Program Files\7-Zip\7z.exe" e -so e:\UnxUtil +sSrc.zip 2>&1 1>nul' or die $@;; Perl> print <PIPE>;; 7-Zip [64] 9.20 Copyright (c) 1999-2010 Igor Pavlov 2010-11-18 Processing archive: e:\UnxUtilsSrc.zip Extracting unxutils Extracting unxutils\CVS Extracting unxutils\CVS\Root Extracting unxutils\CVS\Repository Extracting unxutils\CVS\Entries Extracting unxutils\Downhill Extracting unxutils\Downhill\CVS Extracting unxutils\Downhill\CVS\Root Extracting unxutils\Downhill\CVS\Repository Extracting unxutils\Downhill\CVS\Entries Extracting unxutils\Downhill\DOC Extracting unxutils\Downhill\DOC\CVS Extracting unxutils\Downhill\DOC\CVS\Root Extracting unxutils\Downhill\DOC\CVS\Repository ... Extracting unxutils\zsh\Src\zle_utils.c Extracting unxutils\zsh\Src\zle_utils.pro Extracting unxutils\zsh\Src\zle_vi.c Extracting unxutils\zsh\Src\zle_vi.pro Extracting unxutils\zsh\Src\zle_word.c Extracting unxutils\zsh\Src\zle_word.pro Extracting unxutils\zsh\Src\zsh.exe Extracting unxutils\zsh\Src\zsh.h Extracting unxutils\zsh\Src\zsh.map Extracting unxutils\zsh\Src\ztype.h Extracting unxutils\zsh\Src\_exrc Everything is Ok Folders: 640 Files: 10528 Size: 108861113 Compressed: 38592955 [download] I used -so & >nul to avoid the files actually being created, but omit those and they will. With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday' Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error. "Science is about questioning the status quo. Questioning authority". I knew I was on the right track :) In the absence of evidence, opinion is indistinguishable from prejudice.	[reply] [d/l] [select]
Re^2: Perl and 7zip and backticks by Anonymous Monk on Feb 27, 2016 at 22:43 UTC
Hi Browser. Thanks for your reply, interesting if 7zip updates its progress via STDERR. I tried your example and 7zip seemed to update more often but in blocks of output not each line as it was being zipped (talking large GB) files here so it's easy to see it update per item. I'll fiddle around with your example more. Cheers	[reply]
Re^3: Perl and 7zip and backticks by BrowserUk (Patriarch) on Feb 27, 2016 at 23:13 UTC
but in blocks of output not each line as it was being zipped Unfortunately that is par for the course with pipes; they have their own buffer. Indeed, that pretty much defines what a pipe is: a piece of shared memory buffer space. The pipe will accumulate output from the writer until the buffer (often 4K) fills, and only then does it start to satisfy the reader. In most cases that is a good thing as it prevents the processes giving up time-slices and switching back and forth in lockstep. Other than setting up the pipe yourself -- a messy procedure easy to get wrong -- and using the barely documented ioctl calls to specify the buffer size(s), I don't know any way of avoiding that. With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday' Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error. "Science is about questioning the status quo. Questioning authority". I knew I was on the right track :) In the absence of evidence, opinion is indistinguishable from prejudice.	[reply]
Re: Perl and 7zip and backticks by Corion (Patriarch) on Feb 26, 2016 at 07:39 UTC
For a working implementation of talking to 7zip, see the implementation of the `run` method in Archive::SevenZip (not on CPAN yet, shame on me).	[reply] [d/l]
Re: Perl and 7zip and backticks by hippo (Archbishop) on Feb 26, 2016 at 09:48 UTC
I've tried various flushing techniques with no success. This was discussed earlier in the week and various solutions were suggested there. Did you try those? What were the outcomes? In what way(s) did they fail to meet your requirements?	[reply]
Re^2: Perl and 7zip and backticks by Anonymous Monk on Feb 27, 2016 at 22:40 UTC
Hi Hippo Using the other solutions as per http://www.perlmonks.org/?node_id=1155885 7zip still won't output each line it's zipping in real time. It could be a limitation of 7zip and how it works. Browser suggests that 7zip sends status updates via STDERR you listed an example with STDERR which I did try briefly but couldn't get it to work correctly either, I'll revisit your example if it is correct that 7zip outputs in realtime via STDERR. Cheers	[reply]