in reply to Re: Taking advantage of dual processors
in thread Taking advantage of dual processors [tag://parallel,cygwin,multi-core,fork,i/o]

Well, what I want is maximum throughput for two operations which are currently sequential. It seems like dispatching the database commit to a 2nd cpu would allow the first one to continue reading, but I really dont know.
I have beheld the tarball of 22.1 on ftp.gnu.org with my own eyes. How can you say that there is no God in the Church of Emacs? -- David Kastrup
  • Comment on Re^2: Taking advantage of dual processors

Replies are listed 'Best First'.
Re^3: Taking advantage of dual processors
by sgt (Deacon) on Nov 19, 2007 at 23:01 UTC

    Why don't you try it? make these extra processes happen in a controlled fashion. Even without having a pool of permanent sqlite writers getting input from a queue which would be the right way to do it (as BrowserUK pointed out even though its example uses threads), you can still quickly prototype with the code you've shown using the following: keep an array of n max. commits-to-do and every m <n lines fire m processes system("my_writer $update &"), throttle a bit if ps (or even /proc) gives you too many my_writer processes, increase if your process count drops below a threshold (can be made adaptive...). Decide what to do if you reach n. (slow down, wait for #proc < n_min etc...) cheers --stephan