Re^2: temp hold logfiles in memory?

my diagnosis is perhaps incorrectly based on noticing about 4-5 seconds per minute improvement running the same batch with a 1 megabyte buffer with that loop I posted against the default file handle method.

I'm generally trying to move most of the operation into memory, as the process itself consists of 300 lines of regex match statements for looping through plain text and/or html files.

as for profiling the code, i'm still very much of a novice and am only now branching into code optimisation. if you've got some pointers there it would be greatly appreciated :)

Comment on Re^2: temp hold logfiles in memory?

Replies are listed 'Best First'.
Re^3: temp hold logfiles in memory? by flexvault (Monsignor) on Mar 20, 2011 at 17:41 UTC
You may want to also benchmark the "300 lines of regex match statements". These statements may be taking more time than the I/O, so don't rule them out. For example, could a simple "index" replace some of the regex tests. Also look at the order of the tests, sometimes just reordering the sequence of tests will improve performance. Good Luck "Well done is better than well said." - Benjamin Franklin	[reply]

Replies are listed 'Best First'.

Re^3: temp hold logfiles in memory?
by flexvault (Monsignor) on Mar 20, 2011 at 17:41 UTC

You may want to also benchmark the "300 lines of regex match statements". These statements may be taking more time than the I/O, so don't rule them out. For example, could a simple "index" replace some of the regex tests. Also look at the order of the tests, sometimes just reordering the sequence of tests will improve performance.

Good Luck

"Well done is better than well said." - Benjamin Franklin

[reply]