in reply to Re^2: Using threads to process multiple files
in thread Using threads to process multiple files
Oh. And the final piece of the discussion about IO.
I moved one of the two big files over to my old, slower, fragmented drive and re-ran the same test. Now, despite that reading from the older drive is slower, the times taken to read both files concurrently and sequentially are almost identical even though they are both connected via the same interface:
[15:49:22.32] E:\test>c:piotest.pl big.csv big.tsv big.csv:[167772161] 407.047676086 big.tsv:[100000001] 417.717574120 big.csv:[167772161] 82.103285074 big.tsv:[100000001] 81.984734058 [16:31:59.04] E:\test>c:piotest.pl c:big.csv big.tsv c:big.csv:[167772161] 138.239881039 big.tsv:[100000001] 85.378695965 c:big.csv:[167772161] 141.292586088 big.tsv:[100000001] 83.687027931
|
|---|