in reply to Re^3: IO::Uncompress::Gunzip thread safe?
in thread IO::Uncompress::Gunzip thread safe?

That works pretty well on Linux. It won't work on Windows, but better than nothing. Thanks!
  • Comment on Re^4: IO::Uncompress::Gunzip thread safe?

Replies are listed 'Best First'.
Re^5: IO::Uncompress::Gunzip thread safe?
by BrowserUk (Patriarch) on Nov 21, 2016 at 23:27 UTC
    It won't work on Windows,

    Why not?

    C:\test>dir *.gz Volume in drive C is Local Disk Volume Serial Number is 8C78-4B42 Directory of C:\test 30/12/2014 13:59 228,720 01mailrc.txt.gz 20/03/2009 15:58 73,773,666 chr1.fa.gz 20/03/2009 16:02 11,549,785 chr21.fa.gz 20/03/2009 15:59 54,997,756 chr6.fa.gz 21/11/2016 19:03 13,203 test.gz 5 File(s) 140,563,130 bytes 0 Dir(s) 366,120,411,136 bytes free C:\test>p1 [0]{} Perl> open GZ, 'gunzip -c test.gz |' or die $!; print while <GZ> +;; 1 qwertyuiopasdfghjklzxcvbnm 2 qwertyuiopasdfghjklzxcvbnm 3 qwertyuiopasdfghjklzxcvbnm 4 qwertyuiopasdfghjklzxcvbnm 5 qwertyuiopasdfghjklzxcvbnm 6 qwertyuiopasdfghjklzxcvbnm 7 qwertyuiopasdfghjklzxcvbnm 8 qwertyuiopasdfghjklzxcvbnm 9 qwertyuiopasdfghjklzxcvbnm ...

    With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority". The enemy of (IT) success is complexity.
    In the absence of evidence, opinion is indistinguishable from prejudice.

      I suppose it could work if we had the gzip command on our Windows servers. What provides the gzip command on Windows for you? Security restricts what we can install, but it is worth looking into.

      Also, I noticed that providing the command arguments to open in as a list rather than a single string doesn't work in Windows? Any suggestions for preventing unusual file name characters from causing problems on Linux and Windows?

        I suppose it could work if we had the gzip command on our Windows servers. What provides the gzip command on Windows for you? Security restricts what we can install, but it is worth looking into.

        As the problem you're having with IO::Uncompress::GZip is thread related, there is nothing to stop you creating your own "gzip" command that supports just enough to satisfy your needs.

        Also, I noticed that providing the command arguments to open in as a list rather than a single string doesn't work in Windows?

        Hm Perldoc says: " On Windows, only the system PROGRAM LIST syntax will reliably avoid using the shell; system LIST , even with more than one element, will fall back to the shell if the first spawn fails.". So, as long as you put the command name as the first argument, the rest of the list will be passed as on *nix.

        Any suggestions for preventing unusual file name characters from causing problems on Linux and Windows?
        Personally, I avoid the problem by not using "unusual characters" for filenames; and rejecting them in filenames entered from external sources. Anything else just creates knock on problems for every process that needs to find and use those filenames in down stream. Why bother.

        With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        "Science is about questioning the status quo. Questioning authority". The enemy of (IT) success is complexity.
        In the absence of evidence, opinion is indistinguishable from prejudice.
        7zip can create gzip compatible archives. Another possibility is Git Extensions (GUI for git), which includes several Unixish command line tools incluging gzip.

        Both are open source, so even a code review would be possible.

        As for executing, BrowserUk is right - e.g.
        my @cmd = ( 'c:\Program Files\7-Zip\7z.exe', 'a', '-mx9', '-r', 'alltexts.zip', '*.txt', ); system @cmd;
        would work
        What provides the gzip command on Windows for you?

        UnxUtils (yes, it's a little bit old), maybe Cygwin

        Security restricts what we can install

        UnxUtils does not require any installation, just unpack the binaries from the ZIP archive.

        Strawberry comes with a quite complete build environment, so maybe you could compile gzip from source.

        Alexander

        --
        Today I will gladly share my knowledge and experience, for there are no sweeter words than "I told you so". ;-)
Re^5: IO::Uncompress::Gunzip thread safe?
by Corion (Patriarch) on Nov 22, 2016 at 08:12 UTC

    I've used this approach on both, Windows and unixish OSes with great success. Where did it fail for you and how?