Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl-Sensitive Sunglasses
 
PerlMonks  

Re: Re:x3 It works, it's fast, and it's scalable!

by joealba (Hermit)
on Jan 25, 2002 at 20:00 UTC ( [id://141519]=note: print w/replies, xml ) Need Help??


in reply to Re:x3 It works, it's fast, and it's scalable!
in thread DBI - Oracle 8 - Load many rows into a temp table w/o using SQL*Loader

I thought about that after I posted my response... But I still think that the overhead for all those transactions (even without the commit) would still be higher than a nice, fast bulk load.

I'll benchmark it and post the results here -- after my coffee break. :)

Replies are listed 'Best First'.
Re: Re: Re:x3 It works, it's fast, and it's scalable!
by pmas (Hermit) on May 28, 2002 at 19:36 UTC
    I am *really* curious - how were benchmarks? I am solving almost exactly same problem. Thank you for asking it. Isn't PM great?

    pmas
    To make errors is human. But to make million errors per second, you need a computer.

      Well, some huge (and involuntary) changes have happened since I posted this question... So I can't run benchmarks now to give you real data anymore. But I can tell you that it was a great solution at the time! We beat the heck out of it for 2 months and never had any problems. Our biggest searches ran no more than 3 seconds, with N<1000 searches running at <1 sec.

      But now, I'm in a MySQL environment instead of Oracle. MySQL allows you to input multiple rows in one query as a list. Eliminating the file creation and external executable call to sqlloader can only speed things up more.

      Recently, a few related posts here popped up which had some other good ideas. Super Search is choking on me, so I can't find them right now. If you're still seeking answers, toss a fresh post at SoPW with some specific questions so we can beat this topic around a little again. :)

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://141519]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others drinking their drinks and smoking their pipes about the Monastery: (5)
As of 2024-03-28 20:15 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found