Re^3: Loading 283600 records (Updated)

There are no rules -- beyond minimising the number of opcodes called -- that apply in all situations. Try plugging this into your benchmark:

my %hash;
while( <> ) {
    my( $k, @v ) = unpack( 'a10a10a4', $_ );
    push @{ $hash{ $k } }, \@v
}
[download]

With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.

"Science is about questioning the status quo. Questioning authority".

In the absence of evidence, opinion is indistinguishable from prejudice.

RIP Neil Armstrong

Comment on Re^3: Loading 283600 records (Updated) Download Code

Replies are listed 'Best First'.
Re^4: Loading 283600 records (Updated) by remiah (Hermit) on Sep 23, 2012 at 23:10 UTC
Added 3 test 05 .. unpack, using List::MoreUtils's natatime 06 .. unpack again 07 .. yours Your unpack was faster than mines. This is benchmark results. s/iter 02_split1 05_unpack_natatime 06_unpack_map 04_unpack 03_split2 07_unpack_2 02_split1 6.38 -- -13% -16% -35% -42% -50% 05_unpack_natatime 5.55 15% -- -4% -25% -33% -43% 06_unpack_map 5.34 19% 4% -- -22% -31% -40% 04_unpack 4.18 53% 33% 28% -- -11% -24% 03_split2 3.70 72% 50% 44% 13% -- -14% 07_unpack_2 3.18 100% 74% 68% 31% 16% -- 01_substr 2.70 136% 105% 98% 55% 37% 18% And test code added. Read more... (1233 Bytes) dsheroh told me of in memory SQLite. It's loading time is apparently faster than any of above tests. I will report it later.	[reply] [d/l]
Re^5: Loading 283600 records (substr alias) by Anonymous Monk on Sep 24, 2012 at 03:57 UTC
I tried the substr alias trick from Re^5: Working with fixed length files, and it revealed a problem with your benchmark, the first subs to run are always slower because the disk cache hasn't been filled, and perl.exe hasn't requested all the memory it needs My messy junk :) Read more... (10 kB)	[reply] [d/l]
Re^4: Loading 283600 records (Updated) by remiah (Hermit) on Sep 24, 2012 at 10:31 UTC
Hello, BrowserUK. I posted some benchmarks of sqlite in-memory at the bottom of this thread. It was surprise for me. Please have a look at. and thanks for responding to me. regards.	[reply]