in reply to Optimal way to read in pipe delimited files

Short of moving to an XS based parser, stop making redundant copies of your data ($Record, and @row into a fresh arrayref). 

while( <SRC> ) {
  chomp;
  my( $key, @row ) = split( /\|/, $_ );
  $prod{ $key } = \@row;
}

[download]

Replies are listed 'Best First'.
Re^2: Optimal way to read in pipe delimited files
by bageler (Hermit) on Nov 09, 2005 at 19:49 UTC
    you can take this two steps further by getting rid of $key and using a map: 
    %prod=map{chomp;@a=split/\|/;shift@a,[@a]}<SRC>;

    [download]
    or, expanded, 
    %prod = map {
  chomp;
  @a = split /\|/;
  shift(@a) => [@a]
} <SRC>

    [download]
    You have to do [@a] instead of \@a if you do not localize the array with "my".
[reply]
[d/l]
[select]
      I think that's one step too far. Specifically, getting rid of a descriptive scalar in favor of lumping all the values together and using shift. I don't see any advantage in performance or clarity.
      Caution: Contents may have been coded under pressure.
[reply]
[d/l]

      Could someone please explain the significance of shift@a,[@a] in the above map? I tried removing this statement and it did not work. I really dont know what that statement is doing.
      Also I did not understand why [@a] is to be used if my is not used.
      thanks
      narashima
[reply]
[d/l]
[select]

        Because my gives you a new variable each time through the loop, you can use \@a and all it well because it's a different @a each time. When you don't use my within the loop, it's always the same variable. Thus @a = (1,2,3); $a = \@a; @a =(4,3,9); $b = \@a; will cause $a and $b to both reference the same memory location which will contain the values (4,3,9). [@a] makes a copy of @a and returns a reference to it, so you get a new chunk of memory each time.

        duff

[reply]
[d/l]
[select]
Re^2: Optimal way to read in pipe delimited files
by narashima (Beadle) on Nov 09, 2005 at 19:46 UTC
    Thanks Fletch. But I thought there would something more perlish......
[reply]

In Section Seekers of Perl Wisdom