Beefy Boxes and Bandwidth Generously Provided by pair Networks
There's more than one way to do things
 
PerlMonks  

comment on

( #3333=superdoc: print w/replies, xml ) Need Help??

I've just run a (not yet thoroughly tested; nor cleaned up; nor even fully optimised) solution to my problem, and the results are promising:

C:\test>BiMap.pl -S=5 -F=0.6 -I=33554432 Fetch by Addr & Fetch by Name took 18.535743 seconds for 11881376 item +s in a 33554432 sized BiMap Mem: 965,028 K

For context, here is the single Perl hash solution from the other thread on the same dataset:

C:\test>1112165-hash.pl Start: mem: 7,892 K 23762752 Hash built: mem: 4,382,916 K Fetch by Name took 7.166012 seconds Fetch by Addr took 7.895133 seconds Hash queried: check mem: 4,382,940 K

That equates to just 7% longer to run the same dataset using just over 1/5th the memory. Not bad for a fairly crude POC with still considerable potential for further time reduction.

So, I've expended 3 days -- call it ~50 hrs of my time -- to optimise this.

No matter how you look at it, that is a far better trade-off than your criminally inept suggestion -- to move to a disk-based DB -- which at (a minimum) of 5000x slower, would cost my customer $2800 * 5000 = $14 million for every hour of processing he intends to do.

And given the proposal is to run 100 hours of simulations; the net cost of your suggestion would be $1.4 billion - $280,000 = $1.12 $1.39972 *BILLION* of your customer's money that you would be flushing down the drain; and all because you've eschewed learning anything new for the last 30 years; seemingly preferring to remain more an archaeologist, than a programmer.

I just hope that people will learn from this and finally realise that your malodorously constructed, maliciously provocative, willfully ignorant, hopelessly outdated -- not just non-useful, but perilously inept -- completely worthless, merit-less fluff; is intended to be neither technically useful nor intellectually stimulating; its only purpose being shameful self-promotion of the vilest, any-hits-are-good-hits, form.


With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.

In reply to Re^2: RAM: It isn't free . . .(Mike Robinson:criminally inept.) by BrowserUk
in thread RAM: It isn't free . . . by sundialsvc4

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post; it's "PerlMonks-approved HTML":



  • Are you posting in the right place? Check out Where do I post X? to know for sure.
  • Posts may use any of the Perl Monks Approved HTML tags. Currently these include the following:
    <code> <a> <b> <big> <blockquote> <br /> <dd> <dl> <dt> <em> <font> <h1> <h2> <h3> <h4> <h5> <h6> <hr /> <i> <li> <nbsp> <ol> <p> <small> <strike> <strong> <sub> <sup> <table> <td> <th> <tr> <tt> <u> <ul>
  • Snippets of code should be wrapped in <code> tags not <pre> tags. In fact, <pre> tags should generally be avoided. If they must be used, extreme care should be taken to ensure that their contents do not have long lines (<70 chars), in order to prevent horizontal scrolling (and possible janitor intervention).
  • Want more info? How to link or or How to display code and escape characters are good places to start.
Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others surveying the Monastery: (2)
As of 2022-05-22 12:21 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    Do you prefer to work remotely?



    Results (80 votes). Check out past polls.

    Notices?