Re^2: Taking advantage of dual processors

I'm confused why you think it wouldn't?

Done serially with '.'s representing time taken:

v---------------------------------|
read........munge....insert........
[download]

versus overlapping the insert and the read in parallel:

v-----------------------|
read........munge.....QI.
v-----------------------|
DQ.insert........Wait....
[download]

Even on a single cpu system, and with the DB running on the same box, the insert can go ahead whilst the disk head locates the next chunk of data.

It will depend upon the relative expense of the overlapped operations, but on a multi-cpu system the elapsed time should be reduced?

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.

"Science is about questioning the status quo. Questioning authority".

In the absence of evidence, opinion is indistinguishable from prejudice.

"Too many [] have been sedated by an oppressive environment of political correctness and risk aversion."

Comment on Re^2: Taking advantage of dual processors Select or Download Code

Replies are listed 'Best First'.
Re^3: Taking advantage of dual processors by chromatic (Archbishop) on Nov 19, 2007 at 23:13 UTC
Does SQLite perform blocking flushes on every commit? Does it perform asynchronous writes? Is the file to read on the same bus as the database file? Are they on the same drive? Are they on the same platter? It would surprise me to see much more than a 15% improvement from concurrency, and it wouldn't surprise me at all to see performance decrease.	[reply]
Re^4: Taking advantage of dual processors by BrowserUk (Patriarch) on Nov 20, 2007 at 02:38 UTC
Does SQLite perform blocking flushes on every commit? Does it perform asynchronous writes? Is the file to read on the same bus as the database file? Are they on the same drive? All good questions, but I bet it took me far less time to write the thread code I posted, than it would take the OP to ascertain the answers. And much less than it would take him to a) understand the significance of his findings; and b) construct a formula to determine whether concurrency would be beneficial or not in the light of those findings. It would surprise me to see much more than a 15% improvement from concurrency, Worth having if it's available. And running the benchmark would take very little time. and it wouldn't surprise me at all to see performance decrease. And it's only 10 or so lines of code and say half an hours effort to discard if not. Are they on the same platter? Do modern drives use multiple platters? In any case, it is unlikely that there is any filesystem that would allow you to control the placement of individual files at that level. I must admit that I think there is probably more potential for performance gain in avoiding (at least) 1 of the 3x duplications inherent in the following code: `my $d = Data::Dumper->new([\@dump], ['dump']) ; $d->Purity(1)->Terse(1)->Deepcopy(1); ... ..., $d->Dump);` [download] Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error. "Science is about questioning the status quo. Questioning authority". In the absence of evidence, opinion is indistinguishable from prejudice. "Too many [] have been sedated by an oppressive environment of political correctness and risk aversion."	[reply] [d/l]