in reply to Taking advantage of multi-processor architecture
If your script is performing a lot of IO (reading and writing), consider separating the files onto different physical disks, and importantly don't access files through things like NFS mounts. Sometimes this alone can double the throughput, esp if you are reading and writing two files at the same time on the same physical disk.
You really do not want the system to be swapping while your program is running, since it will cause things to slow down a lot. This is often caused by trying to manipulate massively large data structures in memory in perl. If your script is using lots of memory (e.g. reading two large files completely into memory before processing), consider processing as you read each line in. If you need them in arrays, consider using something like Tie::File. One common example is trying to sort a massive array within perl itself. Sometimes you can call an external utility to do this for you much more quickly (e.g. GNU sort).
If you give more details, I'm sure someone can help out more.
|
|---|