We are updating 400 to 500 thousand rows of data a week. our update takes about 2 days to finish.
That looks very slow.
But you don't give any useful information - perhaps 2 days is the best you can get. Some question to start with:
- What database?
- What queries?
- How many queries?
- How do the tables look like?
- Which indices are there?
- How large are the tables?
- How many other processes access the tables?
- Where's the time spend, in the database, or the program collecting the data?
- How much of the spend time are you waiting to get locks?
- Are you using transactions?
- Are other processes using transactions?
- Are you sure it's a Perl questions, and not a database/SQL question, better answered in a forum with more expertise?