How to sort data in the input file ?

bh_perl has asked for the wisdom of the Perl Monks concerning the following question:

Hi,

As your information, I have a text file which are contain a sample data as below:-

1&20070102&string3&string4....
2&20070101&string3&string4....
3&20061212&&string3&string4....
[download]

Based on the sample data, I want to open the text file and sort the data based on the date (second &-delimited field) and the expected output data would be:-

3&20061212&&string3&string4....
2&20070101&string3&string4....
1&20070102&string3&string4....
[download]

Could somebody help me how to do this ?

Previously I have write the program as below but its not working on the last part, why this case happened ?

 opendir(INDIR, $outdir) or die ("$indir not found\n");
        while (my $infile = readdir(INDIR)) {
                chomp;
                next if $infile !~ /postpaid/;

                #change & to space for sorting purpose
                qx(cat $outdir/$infile |sed 's/\&/\t/g' > $outdir/$inf
+ile.tmp);

                #sort the data based on date or second colum
                qx(cat $outdir/$infile.tmp |sort -2 > $outdir/$infile.
+srt);

                #change back the space to &
                #NOTE: This one not working
                qx(cat $outdir/$infile.srt |sed 's/\t/\&/g' > $outdir/
+$infile.ok);

                #remove temporary file
                qx(rm -f $outdir/$infile.tmp);
        }
        closedir(INDIR);
[download]

Comment on How to sort data in the input file ? Select or Download Code

Replies are listed 'Best First'.
Re: How to sort data in the input file ? by kyle (Abbot) on Feb 09, 2007 at 02:32 UTC
You seem to be a lot more comfortable with shell programming than Perl programming. I think you could do all of what you want in the shell like so: `for file in $outdir/postpaid do sort -t '&' -k 2 < $file > $file.ok done` [download] You can also do it all in Perl, and there are many ways to accomplish that. This is probably one of them: `opendir(INDIR, $outdir) or die "Can't opendir $outdir: $!\n"; while (my $infile = readdir(INDIR)) { next if $infile !~ /postpaid/; open my $infile_fh, '<', "$outdir/$infile" or die "Can't read $infile: $!\n"; my @lines = <$infile_fh>; close $infile_fh; open my $outfile_fh, '>', "$outdir/$infile.ok" or die "Can't write $infile.ok: $!\n"; print $outfile_fh sort { ($a1) = ($a =~ /^\d+\&(\d{8})\&/); ($b1) = ($b =~ /^\d+\&(\d{8})\&/); $a1 <=> $b1 } @lines; close $outfile_fh; } closedir(INDIR);` [download]	[reply] [d/l] [select]
Re: How to sort data in the input file ? by fmerges (Chaplain) on Feb 09, 2007 at 02:45 UTC
Hi, One solution could be: `my @sorted_lines = map { $_->[1] } sort { $a->[0] <=> $b->[0] } map { [(substr $_,2,8) => $_] } <DATA>; __DATA__ 1&20070102&string3&string4.... 2&20070101&string3&string4.... 3&20061212&&string3&string4....` [download] If your files are really big, then you could try another approach... Regards, fmerges at irc.freenode.net	[reply] [d/l]
Re: How to sort data in the input file ? by ww (Archbishop) on Feb 09, 2007 at 02:20 UTC
One problem may be that your data is INconsistent. Note the doubled "&" in 3 after the date. More generally, this looks like a case where you might profitably use a different regex to capture the just the date (eg `m%\d&(\d{8})&%` ) and a hash, keyed on the date, with the full string as a value. Once you have it in the proposed hash, sorting comes easy...	[reply] [d/l]