I would try to do as much work in the database as possible. In this case, I would create a (temporary) table with a single column containing the PK values, and join on that (which effectively filters your results). However, you are using mysql and AFAIK that database is able to do only nested-loop joins instead of the join modes far more fitting for that sort of query, so this method might actually be worse performance-wise. If the PK text file doesn't change and you need to run this query repeatedly, it may be a convenient method. Adding a column to the table containing whether it is "allowed" may be acceptable, too.
Your other option would be to emulate the hash join strategy with Perl's hashes and filter manually. There's nothing wrong with that, but it does require quite a bit of data transferred out of the database. Do note that if you do it a single record at a time, it takes little memory, as the largest thing you'll need is a hash made out of the PKs in FILE1. (Listen to the other posters, too; they make good points)
In reply to Re: Alternative to querying a database to save time
by Anonymous Monk
in thread Alternative to querying a database to save time
by smandape1
| For: | Use: | ||
| & | & | ||
| < | < | ||
| > | > | ||
| [ | [ | ||
| ] | ] |