in reply to Threading - getting better use of my MP box
Where exactly is your bottleneck? Add debugging information that keeps track of the time used in each of the three primary functions of this program: downloading of data, parsing of data, inputting of data into the database. I presume that the most likely place where your program is using the most time would be the downloading of information. Therefore look into creating a unique thread for downloading from each source of data.
Most likely the parsing and inputting of data into the database are the quickest operations by far, so you can leave these as single threaded for simplicity sake.
- Miller