pglinx has asked for the wisdom of the Perl Monks concerning the following question:

Anybody know how this can be done?

Read in file 1 and output to file 2

file 1
848	05/23/06	11:00
848	05/23/06	12:30
848	05/23/06	13:00
848	05/23/06	14:00
848	05/25/06	11:00
848	05/25/06	12:00
261	05/24/06	11:00
261	05/24/06	12:30
261	05/24/06	13:00
261	05/24/06	13:00
261	05/24/06	13:00
261	05/24/06	13:00

file 2
848	05/23/06	11:00,12:30,13:00,14:00
848	05/25/06	11:00,12:30
261	05/24/06	11:00,12:30,13:00

Replies are listed 'Best First'.
Re: Combining flat file records
by chromatic (Archbishop) on May 08, 2006 at 22:26 UTC

    What part is giving you trouble?

    If I were writing this, I'd read a line, split on whitespace, and use the first two fields as keys of a hash of array references. If order matters, I'd keep another array for order of the hash keys (or maybe use something like Tie::IxHash).

    Producing the right output is just a matter of a loop, a join, and a print.

Re: Combining flat file records
by jZed (Prior) on May 08, 2006 at 22:43 UTC
    Perhaps this is fishing for you rather than teaching you to fish, but here's how I'd do it:
    #!/usr/bin/perl -w use strict; my %result; for my $row(<DATA>) { my($num,$date,$time)=split /\s+/,$row; push @{ $result{$num}->{$date} }, $time; } for my $num(sort keys %result) { for my $date(sort keys %{$result{$num}}) { my %contains; my @time_ary; for my $time(sort @{$result{$num}->{$date}} ){ push @time_ary, $time unless $contains{$time}++; } printf "%s\t%s\t%s\n", $num,$date,join(',',@time_ary); } } __DATA__ 848 05/23/06 11:00 848 05/23/06 12:30 848 05/23/06 13:00 848 05/23/06 14:00 848 05/25/06 11:00 848 05/25/06 12:00 261 05/24/06 11:00 261 05/24/06 12:30 261 05/24/06 13:00 261 05/24/06 13:00 261 05/24/06 13:00 261 05/24/06 13:00

      If speed or memory is an issue, one could avoid loading the entire file into memory as follows:

      my $last_id; my $last_date; my @times; for (;;) { my $line = <DATA>; my ($id, $date, $time) = split(' ', $line||''); if ($. != 1) { if ((!defined($id) && @times) || $id ne $last_id || $date ne $last_date ) { print(join("\t", $last_id, $last_date, join(',', @times)), "\ +n"); $#times = -1; } } last unless defined($line); $last_id = $id; $last_date = $date; push(@times, $time) if !@times || $times[-1] ne $time; } __DATA__ 848 05/23/06 11:00 848 05/23/06 12:30 848 05/23/06 13:00 848 05/23/06 14:00 848 05/25/06 11:00 848 05/25/06 12:00 261 05/24/06 11:00 261 05/24/06 12:30 261 05/24/06 13:00 261 05/24/06 13:00 261 05/24/06 13:00 261 05/24/06 13:00

      It also preserves the order of the ids in the output.

        Um, yeah. I was being lazy. Better to get it all in one pass as you show.
      Thank you, I new how to loop through a file using foreach to reformat the output, but was not sure how to actually combine it, especially in one field. Thanks for the help and for teaching me how to fish. It worked great.
Re: Combining flat file records
by GrandFather (Saint) on May 09, 2006 at 00:25 UTC

    Another way to do it:

    use strict; use warnings; my %data;
    map {chomp $_->[2]; $data{$_->[0]}{$_->[1]}{$_->[2]} = ''} [split /\s+ +/] for <DATA>;
    $data{$_->[0]}{$_->[1]}{$_->[2]} = '' for map [split], <DATA>; for my $num (reverse sort keys %data) { print "$num\t$_\t", (join ",", keys %{$data{$num}{$_}}), "\n" for keys %{$data{$num}}; } __DATA__ 848 05/23/06 11:00 848 05/23/06 12:30 848 05/23/06 13:00 848 05/23/06 14:00 848 05/25/06 11:00 848 05/25/06 12:00 261 05/24/06 11:00 261 05/24/06 12:30 261 05/24/06 13:00 261 05/24/06 13:00 261 05/24/06 13:00 261 05/24/06 13:00

    Prints:

    848 05/23/06 12:30,11:00,14:00,13:00 848 05/25/06 11:00,12:00 261 05/24/06 12:30,11:00,13:00

    Update: apply jwkrahn's neat round of golf


    DWIM is Perl's answer to Gödel
      map {chomp $_->[2]; $data{$_->[0]}{$_->[1]}{$_->[2]} = ''} [split /\s+ +/] for <DATA>;
      Could be written more simply as:
      map {$data{$_->[0]}{$_->[1]}{$_->[2]} = ''} [split] for <DATA>;
      Or without using map in a void context:
      $data{$_->[0]}{$_->[1]}{$_->[2]} = '' for map [split], <DATA>;
      :-)
Re: Combining flat file records
by sgifford (Prior) on May 08, 2006 at 22:28 UTC
    Smells a little like homework; is it? If not, can you explain briefly what you're doing this for?
Re: Combining flat file records
by jwkrahn (Abbot) on May 08, 2006 at 23:58 UTC
    Sure.
    #!/usr/bin/perl use warnings; use strict; my %data; while ( <DATA> ) { my ( $key, $time ) = /([^\t]+\t[^\t]+)\t(.+)/ or next; if ( eof or not exists $data{ $key } ) { print join( "\t", %data ), "\n" if %data; %data = ( $key, $time ); } else { $data{ $key } = join ',', $data{ $key }, $time unless $data{ $key } =~ /$time/; } } __DATA__ 848 05/23/06 11:00 848 05/23/06 12:30 848 05/23/06 13:00 848 05/23/06 14:00 848 05/25/06 11:00 848 05/25/06 12:00 261 05/24/06 11:00 261 05/24/06 12:30 261 05/24/06 13:00 261 05/24/06 13:00 261 05/24/06 13:00 261 05/24/06 13:00