in reply to Re: How can I read multiple lines starting with the same number and put in to a nested array and print it to a file?
in thread How can I read multiple lines starting with the same number and put in to a nested array and print it to a file?

The initial input file can be 3 GB. I have a NASTRAN (Finite Element Analysis software) output file which is pre sorted. The first number in each line represent an element. Element number can be aywhere between 1 to 9999999. They are not consistent either. The example above was just a simplified example. I would not know exact element numbering. So I might have 10 lines starting with element ID 12000. The next element id might be 13450 and I might have 10 lines starting with that number. So I have a huge file, and memory is crucial. I need to have an inteligent algorithm that goes through each line and finds the lines starting with the same number (which I would not know what they are) and put only those ones in to an array. I do not need to create different array for each element number. Once I print the sorted nested array, I can use the same array name and put new lines starting with the same numbers into this array. I am not sure if I am making sense. But thanks for listening.
  • Comment on Re: Re: How can I read multiple lines starting with the same number and put in to a nested array and print it to a file?

Replies are listed 'Best First'.
Re: Re: Re: How can I read multiple lines starting with the same number and put in to a nested array and print it to a file?
by BrowserUk (Patriarch) on Jun 24, 2002 at 08:54 UTC

    This is untested and bad Perl (I'm new to it) but the algorithm should be clear and work ok. If your lucky, one of the experts here will be so appalled by my Perl that he will step in a clean it up or offer you better.

    # somewhere to remember the records we processing my $lastFirstNum = ""; my @nums; # work array while (<>) { # get the first number from the line my $firstNum = split /\s/, $_, 1; # prime the pump if its the first time through $lastFirstNum = $firstNum if $lastFirstNum = ""; if ( $firstNum eq $lastFirstNum ) { # its still the same type so save it push @nums, $_; next; # skip to next record } # we found the last one sort the array @nums = map {[ reverse sort @$_ ]} @nums; #open output using the first number as the name open( FH, ">$lastFirstNum" ) or die "Can't open $lastFirstNum: $!" +; print @nums; close( FH ) or die "Couldn't close $lastFirstNum: $!"; # the number changed $lastFirstNum = $firstNum; undef @nums; #clean the array push @nums, $_; # push the new record }

    Update: corrected my own (first) obvious mistake.