in reply to Basic Perl array intersection faster than mysql query join.

Be careful: you are heading for trouble. There are various reasons why joining in Perl will sometimes be faster than a database. One might be a poor choice of database, or an obsolete version. Another might be missing indexes or incorrect SQL. There are also many tuning options on most databases.

The most obvious thing though is that in order to do this in Perl you have to retrieve all of the records that you might possibly be interested in so that you can examine them. In any database of non-trivial size, that would be a very slow operation. In short, your in-memory approach will lose to the database as soon as the amount of data to be examined reaches a point where fetching it all from the database is too slow.

  • Comment on Re: Basic Perl array intersection faster than mysql query join.

Replies are listed 'Best First'.
Re^2: Basic Perl array intersection faster than mysql query join.
by zakzebrowski (Curate) on Oct 18, 2004 at 13:56 UTC
    Also, speed isn't the only thing to consider... If you are ever going to alter the tables or multiple table joins, or try to add full text indexing, or try to do something more complicated, a database will help you. Using a database versus using a (insert language here) script to do what you need to do, is a tradeoff between speed, maintainability, and east of expandibility... If, though, you know that all you will be ever doing is joining those two tables, and you are ready to deal with the risk of doing the join in the code base, and you need the speed, go for it.


    ----
    Zak - the office
[reply]

In Section Seekers of Perl Wisdom