in reply to Re: Large file, multi dimensional hash - out of memory
in thread Large file, multi dimensional hash - out of memory

great idea. yes the data is sorted. How can I dump that data to a file and free up the hash so I don't run out of memory?
  • Comment on Re^2: Large file, multi dimensional hash - out of memory

Replies are listed 'Best First'.
Re^3: Large file, multi dimensional hash - out of memory
by davido (Cardinal) on May 15, 2013 at 15:23 UTC

    Here's an example of what I was suggesting.

    my $current_name = ''; my %extensions = (); while ( my $line = <DATA> ) { chomp $line; my( $name, $ext ) = split ' ', $line; if( $name ne $current_name ) { print "($current_name) => [", join( ', ', sort keys %extensions ), + "]\n" if length $current_name; $current_name = $name; %extensions = (); } $extensions{$ext}++; } print "($current_name) => [", join( ', ', sort keys %extensions ), "]\ +n"; __DATA__ /foo/bar/baz/123 aaa /foo/bar/baz/123 aab /foo/bar/baz/123 aac /foo/bar/baz/124 aaa /foo/bar/baz/124 aab

    The memory footprint will only be as big as the maximum number of extensions per name.


    Dave