I need to write a perl script to read gzipped fastq files from a text file list of their paths and then concatenate them together and output to a new gzipped file. ( I need to do this in perl as it will be implemented in a pipeline) I am not sure how to accomplish the zcat and concatenation part, as the file sizes would be in Gbs, I need to take care of the storage and run time as well.

So far I can think of it as - 
use strict;
use warnings;
use IO::Compress::Gzip qw(gzip $GzipError) ;

#-------check the input file specified-------------#

$num_args = $#ARGV + 1;
if ($num_args != 1) {
    print "\nUsage: name.pl Filelist.txt \n";
exit;

$file_list = $ARGV[0];

#-------------Read the file into arrray-------------#

my @fastqc_files;   #Array that contains gzipped files 
use File::Slurp;
my @fastqc_files = $file_list;


#-------use the zcat over the array contents 
my $outputfile = "combined.txt"
open(my $combined_file, '>', $outputfile) or die "Could not open file 
+'$outputfile' $!";

for my $fastqc_file (@fastqc_files) {

    open(IN, sprintf("zcat %s |", $fastqc_file)) 
      or die("Can't open pipe from command 'zcat $fastqc_file' : $!\n"
+);
    while (<IN>) {
        while ( my $line = IN ) {
          print $outputfile $line ;
        }
    }
    close(IN);

my $Final_combied_zip = new IO::Compress::Gzip($combined_file);
  or die "gzip failed: $GzipError\n";

[download]
Somehow I am not able to get it to run. Can anyone share if there is simpler/ correct method to accomplish this? Thanks!

In reply to concatenate/stitch multiple GZip fastq files and output combined gzip file by Bioinfocoder

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.