in reply to Re: Threads From Hell #2: How To Parse A Very Huge File
in thread Threads From Hell #2: How To Search A Very Huge File [SOLVED]
And then there's:
$ time grep karl very_huge.file nose cuke karl real: 2.127s == user: 1.845s + sys: 0.283s ## CLOSE ENOUGH! $ time ./MCE-1.608/bin/mce_grep karl very_huge.file nose cuke karl real: 1.061s != user: 2.176s + sys: 1.616s ## NOT EVEN CLOSE to: 3. +792s $ time ./MCE-1.608/examples/egrep.pl karl very_huge.file nose cuke karl real: 0.690s != user: 2.165s + sys: 0.362s ## NOR IS THIS CLOSE to: + 2.527s
Looks dodgy to me.
And then the claim that:
The following code snippet parses the 2 GiB file in 1 second.
Let's examine that. The code: slurps the entire 2GiB file into memory and then starts 4 workers that each get a reference to (1/4 of???) the slurped data and then:
So the regex is run against 1.25 times as much data as is contained in the file, and takes "less than one second"; which is less than the 2.127 seconds the real grep takes, despite that it only processes the file's data once?
Have you heard the song: Funny numbers?
|
---|
Replies are listed 'Best First'. | |
---|---|
Re^3: Threads From Hell #2: How To Parse A Very Huge File
by marioroy (Prior) on May 24, 2015 at 13:27 UTC |