Beefy Boxes and Bandwidth Generously Provided by pair Networks
No such thing as a small change
 
PerlMonks  

Re^3: performance of File Parsing

by sundialsvc4 (Abbot)
on Jul 07, 2011 at 12:26 UTC ( [id://913172]=note: print w/replies, xml ) Need Help??


in reply to Re^2: performance of File Parsing
in thread performance of File Parsing

What’s killing you, then, is “that enormous hash.”   You need to replace that logic.

If you were to plot the throughput of this program, it would describe a nice, exponential curve.   When it reaches the “thrash point,” it smashes into the wall and drops dead.   That’s my blindfolded prediction, but I’ll bet I’m right on the money.

I suggest stuffing the whole thing into an SQLite database (flat-file), and using queries (within transactions).

“Don’t ‘diddle’ the code to make it faster ... find a better algorithm.”
– Kernighan & Plauger; The Elements of Programming Style.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://913172]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others meditating upon the Monastery: (5)
As of 2024-03-29 07:51 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found