However, after read all data into memory, I need to process it from beginning to end once to get wanted data line based on given criteria,
Why read it all in -- ie. read every line, allocate space for every line, extend the array to accommodate every line -- if you only need to process the array once?
In other words, why not?:
while( <TEMPFILE> ) { processLine( $_ ) }
Also, you are throwing away performance with how you are passing data back from your subroutines. Eg:
return (%hashed);
That builds a hash in hash_array(), the return statement converts it to a list on the stack; then back at the call site:
my (%trec) = &hash_array(@arr);
You convert that list from the stack back into another hash. Then, you immediately return that hash to the caller of line2rec(), converting it to another list on the stack:
my (%trec) = &hash_array(@arr); return (%trec); }
And then back at that call site, you convert that list back into yet another hash:
my (%trec) = &line2rec($line);
And all of that in order to test if the line contains the string 'active':
if ($trec{'active'})
my @data; while( <TEMPFILE> ) { /active/ and push @data, $_; }
It'll be *much* faster.
In reply to Re: Process large text data in array
by BrowserUk
in thread Process large text data in array
by hankcoder
| For: | Use: | ||
| & | & | ||
| < | < | ||
| > | > | ||
| [ | [ | ||
| ] | ] |