Lhamo_rin has asked for the wisdom of the Perl Monks concerning the following question:

I am writing a perl script to merge several dozen data files into one file. The merger is pretty easy but after the merger I have to change a line for each data snapshot so that they are sequential. For example, my merged data file would look something like: 
"1"
{
     "time: 1247";
     "sump:  15";
}
"2"
{
     "time: 1255";
     "sump:  9";
}
"8"
{
     "time: 1350";
     "sump:  7";
}
"5"
{
     "time: 1600";
     "sump:  3";
}

[download]
I have to re-order the "1", "2", "5", and "8" to be "1", "2", "3", and "4". I have to do this for several thousand points. Any ideas how I can do this most efficiently?

Replies are listed 'Best First'.
Re: Re-ordering data in data file
by tlm (Prior) on Jun 27, 2005 at 18:57 UTC

    This is the basic idea of what I'd do: 

    my $count = 0;
while (<IN>) {
  s/^"\d+"$/'"' . ++$count . '"'/e;
  print OUT;
}

    [download]
    ...or, from the command line 
    % perl -i.bak -pe 's/^"\d+"$/q(").++$count.q(")/e' myfile

    [download]
    The one-liner puts the original file in myfile.bak.

    the lowliest monk

[reply]
[d/l]
[select]
Re: Re-ordering data in data file
by sk (Curate) on Jun 27, 2005 at 19:01 UTC
    Do you want to re-arrange the blocks or just rename them? If you just want to rename them then use tlm's code. If you want to move the blocks around then it would be good to understand how these blocks are merged into one file. It makes more sense to merge them in the way you want than merging all and then moving them around.

    cheers

    SK

[reply]
      I'm sorry I should have been more clear. I just need to re-number, not re-order the data.
[reply]
Re: Re-ordering data in data file
by ikegami (Patriarch) on Jun 27, 2005 at 19:01 UTC
    Here's a minimalistic solution: 
    while (<DATA>) {
   s/^"\d+"/'"' . ++$i . '"'/e;
   print;
}

__DATA__
"1"
{
     "time: 1247";
     "sump:  15";
}
"2"
{
     "time: 1255";
     "sump:  9";
}
"8"
{
     "time: 1350";
     "sump:  7";
}
"5"
{
     "time: 1600";
     "sump:  3";
}

    [download]

    As a one-liner:

    perl -i.bak -pe 's/^"\d+"/++$i; qq{"$i"}/e;' file          <- Unix
perl -i.bak -pe "s/^"""\d+"""/++$i; qq{"""$i"""}/e;" file  <- Windows

    [download]
[reply]
[d/l]
[select]

      I'm not sure if I should post this since it isn't a reply to the OP Question but it might be intresting (or not)...

      When I write a one-liner, which should run on multiple platforms or when I'm unsure of the platform (for example when posting it somewhere), then I never use ' or " in the code.

      Instead of using them I use qq and q, and I use the hex-values (\x22 for ", and \x27 for '). The advantage of this is that someone should only need to change the outer symbols.

      This turns your one-liner into:

      perl -i.bak -pe 's/^\x22\d+\x22/++$i; qq{\x22$i\x22}/e;' file

      [download]

[reply]
[d/l]
Re: Re-ordering data in data file
by TedPride (Priest) on Jun 27, 2005 at 19:53 UTC
    Obviously, the records need to be reordered along with being renumbered. As far as I can tell, the above "solutions" don't work. 
    use strict;
use warnings;

my ($c, %hash);
$_ = join '', <DATA>;
$hash{$1} = $2 while m/"(\d+)"(.*?{.*?})/gs;
print "\"".++$c."\"$hash{$_}\n" for sort {$a <=> $b} keys %hash;

__DATA__
"1"
{
     "time: 1247";
     "sump:  15";
}
"2"
{
     "time: 1255";
     "sump:  9";
}
"8"
{
     "time: 1350";
     "sump:  7";
}
"5"
{
     "time: 1600";
     "sump:  3";
}

    [download]
[reply]
[d/l]
Re: Re-ordering data in data file
by graff (Chancellor) on Jun 28, 2005 at 01:35 UTC
    I'd go for setting $/ (input_record_separator) so that you read a whole logical record at a time; that makes the numbering and editing a lot easier: 
    {
    local $/ = "}\n";
    while (<>) {
        s/^"\d+"/"$."/;
        print;
    }
}

    [download]
    (This is based on your later clarification that the records themselves do not need to be re-ordered according to the original record numbers before assigning the new sequential record numbers.)
[reply]
[d/l]

Back to Seekers of Perl Wisdom