Re^2: Working with a very large log file (parsing data out)

... and be substantially faster (~1.25 hours)

How did you figure the time?

Comment on Re^2: Working with a very large log file (parsing data out)

Replies are listed 'Best First'.
Re^3: Working with a very large log file (parsing data out) by BrowserUk (Patriarch) on Feb 20, 2013 at 08:49 UTC
By running it on a 5.4GB logfile -- that took 12.5 minutes -- and then scaling: 1.5TB / 5.4GB = 285 * 12.5 = 35550 / 60 = 59.2. + (a bit for contingency) = 75. And then making the mistake of treating that as minutes instead of hours! Thank you for the heads up, I'll correct the above! With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday' Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error. "Science is about questioning the status quo. Questioning authority". In the absence of evidence, opinion is indistinguishable from prejudice.	[reply]

Replies are listed 'Best First'.

Re^3: Working with a very large log file (parsing data out)
by BrowserUk (Patriarch) on Feb 20, 2013 at 08:49 UTC

By running it on a 5.4GB logfile -- that took 12.5 minutes -- and then scaling: 1.5TB / 5.4GB = 285 * 12.5 = 35550 / 60 = 59.2. + (a bit for contingency) = 75.

And then making the mistake of treating that as minutes instead of hours!

Thank you for the heads up, I'll correct the above!

With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.

"Science is about questioning the status quo. Questioning authority".

In the absence of evidence, opinion is indistinguishable from prejudice.

[reply]