Hi all,
I'm in the decision phase of writing a high demanding application. The most demanding task is related with (approximate) string matching algorithms (suffix trees, dynamic programming, edit distances, maybe eulerian paths, etc...). Previous related applications (mostly written >5 years ago) were written in high performance programming languages like C/C++, but I wonder if today's muti-core architecture opens a new way of facing this kind of applications (through concurrent programming languages).
Trying to solve this question, I've been checking current possibilities in concurrent programming languages and I've been surprised by the overwhelming number of alternatives that are being developed
Functional programming style seems to facilitate the development of concurrent programs (no side effects). Among these, Erlang and its asynchronous message passing model is probably the one that is getting more attention, but there are other examples like Haskell, Clojure (acts through the JVM), Alice or Clean.
There are more alternatives out there not stuck in the functional paradigm like Scala (which runs on the JVM), cilk (concurrent C), Oz/Mozart, occam... and over 20 more possibilities!!. Not counting recent develops like Fortress, Chapel or IBM's X10
Wow... there are many possibilities, aren't they? Some questions (most of them probably without answer) comes to mind:
- Is multicore programming the near-mid future of programming? and if so, is Perl doing any effort (maybe trough Perl6) in that direction?
- Will the functional programming languages become more relevant with the multi-core revolution?
- In terms of program execution speed, will it worth to choose a "slower" programming language that allows easy (built-in) concurrent programming over "fast" single-core programming language?
- Will it worth to start today my project in one of those concurrent programming languages?
- Which ones will survive, which will be abandoned and which will be a reference for concurrent programming?
As always, any comment would be highly appreciated
citromatik
|
---|
Replies are listed 'Best First'. | |
---|---|
Re: (OT) Programming languages for multicore computers
by moritz (Cardinal) on May 04, 2009 at 13:53 UTC | |
Is multicore programming the near-mid future of programming?IMHO, yes and if so, is Perl doing any effort (maybe trough Perl6) in that direction? Yes. Perl 6 is specified with concurrency in mind, and there are some constructs that can be automatically parallelized, and many features are geared to make that painless. Sadly only pugs implements some parallelism so far, and only very basic constructs. Will the functional programming languages become more relevant with the multi-core revolution? Especially pure functional languages like Haskell. I can imagine it's much easier for the compiler to determine interdependencies than for normal, imperative languages. But it remains to be seen how much that will matter in practice; I believe that both language families have much space to improve wrt concurrency. Will it worth to start today my project in one of those concurrent programming languages? That really depends on how easy or hard they are to parallelize. I'm currently doing some simulations that are embarrassingly simple to parallelize - I always have to cover a large parameter space, so I just start one process per parameter, let it run for an hour and collect the result in a file - there's really no point parallelizing that more unless somebody throws more than 200 CPUs at me, each equipped with at least 8GB RAM. Which ones will survive, which will be abandoned and which will be a reference for concurrent programming? I don't think that's clear yet. There are a few language that I'm pretty sure will stay, which have proven to fit very well for some problems (C and Haskell for example; I'm also pretty sure that Erlang has a sufficiently large community to survive). | [reply] [Watch: Dir/Any] |
by shmem (Chancellor) on May 05, 2009 at 20:02 UTC | |
Sadly no only pugs implements some parallelism so far, and only very basic constructs. But having that in mind: it should be done in the core, at the very bottom, since it affects all that comes after in terms of implementation. | [reply] [Watch: Dir/Any] |
by moritz (Cardinal) on May 05, 2009 at 20:23 UTC | |
| [reply] [Watch: Dir/Any] |
Re: (OT) Programming languages for multicore computers
by samtregar (Abbot) on May 04, 2009 at 16:45 UTC | |
I'm very sweet on Erlang, but whether it's right for you is something only you can judge. You might get to market faster by leaning on the built-in parallel features or you might get bogged down in the learning curve of a new environment. I think a lot depends on just how much parallelism you need to exploit - if we're talking about dual and quad core machines then some fork()ing Perl code is probably good enough. If you're deploying on a cluster of several hundred 32-core machines, well, you might need more management ifrastructure and going with a pre-built system may be a better idea than rolling your own. -sam | [reply] [Watch: Dir/Any] |
Re: (OT) Programming languages for multicore computers
by BrowserUk (Patriarch) on May 06, 2009 at 10:07 UTC | |
The following was started 2 days ago, but I've been unable to finish it. It was suggested to me I post it as-is anyway, and if there is sufficient interest, I'll try to complete it later. My first response to almost every question you've asked, is: It depends. It mostly depends upon the requirements of the algorithm and/or data that needs processing. There are essentially 3 basic types of parallelism (with many variations): The future is threaded. We are just waiting for the software to catch up with the hardware. And I fear we are going to have to wait for the next generation of programmers before that problem will be properly fixed. Just as many of my generation have problems with using SMS--and with the language forms it generates--whilst it is simpler than tying shoelaces for the new generations. So it is with threading. Many in my generation only see the problems involved--not the potential. Just as it tool the ubiquity of the web to drive the transition from th request-response model to the upcoming Web 2 era. So, it will require the ubiquity of threads and cores to drive the transition from forks&pipes to threads&shared state.It may take until the retirement of the Luddite generation for it to happen. But it will happen. As you've rightly indicated, one of the primary drivers of the transition will be the evolution of computer languages that give easy--and well abstracted--access to the potential. Whilst many of those you cite are a adept at one or more of the types of concurrency, none of them are truly adept at all of them. And problems arise because real-world problems tend to require two or more of those types in the same application. A secondary problem with most existing language abstractions of concurrency is that they tend to take one of two tacts: The solution is relatively simple, in concept at least. There needs to be a clear delineation between that state that needs to be shared--for efficient concurrency. And that state that mustn't be shared--for algorithmic safety. And the programmer must have explicit control over the former, whilst the compiler ensures and enforces the latter. In this way, thread procedures can be written without regard to the safety of local variables and state. Anything declared 'local', or 'non-shared', or better, any not explicitly marked as shared, is protected through compiler-enforced mechanisms from leaking between threads. At the same time, shared state can be passed around between threads as the needs of the algorithm dictate, without incurring huge penalties of copying involved in write-once variables. Another way of viewing the two concurrency abstractions is: There is a third answer to the problems of locking and synchronisation. Avoid them! Sounds too simple to be a point worth making, but it is a fact that most algorithms and applications that can benefit from concurrency can, with a little care, be architected such that they use little or no locking or synchronisation, despite that they may manipulate large volumes of shared state, in place. The trick is to program the application so that only one thread attempts to access any given piece of data an any given time. I'm not talking about using mutexs to prevent concurrent access. More, mechanisms that don't allow the programmer to try. But without throwing the (procedural) baby out with the bath water. This can be done today--in any language. It just requires the programmer to be aware of the techniques and apply them. But it would be a whole lot easier if programmers didn't have to become aware of the implementation details or re-invent them every time. To that end, there are several new abstractions that can help: With a few simple control and data sharing constructs such as these, all the scary, difficult parts of threading disappear under the covers, and the programmer can get back to concentrating uponn their application whilst knowing that it will benefit from whatever core and threads are available. You can imagine the pipeline example (read; decode; transform; encode; write), from above being written something along the lines of;
Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.
| [reply] [Watch: Dir/Any] [d/l] [select] |
by tilly (Archbishop) on May 07, 2009 at 04:59 UTC | |
The short version of what I said there is that Moore's law is looking to add cores faster than SMP can scale, at which point we'll go to NUMA inside of a CPU. And once we go to NUMA, multi-threaded programs don't scale. So while there are some very important niches which can benefit from serious multi-threading, I don't buy the dream of unlimited numbers of cores available to a single multi-threaded application. | [reply] [Watch: Dir/Any] |
by BrowserUk (Patriarch) on May 07, 2009 at 14:12 UTC | |
Thank you. That's an interesting article to read. But, like so many articles that set out to prove the point they want to make, rather than looking at the evidence and see where it leads, you spend much of the time setting up Straw Men, and then proceed to knock them down. (**) My prediction for the medium-term 10 year, future (I chickened out of the long-term :), is that each application will be a single, multi-threaded process running within its own virtualised OS image. And at the hypervisor level, the task will be to simply distribute the threads and physical memory amongst the cores; bringing more cores on-line, or switching them off, as the workloads vary. This means that the threads of a particular VM/process/application will tend to cluster over a limited number of the available cores, controlled by dynamic affinity algorithms, but with the potential to distribute any single task's threads over the full complement of available cores should the need arise, and appropriate algorithm be chosen. But that's about as far as I'm prepared to go. Time will tell for sure :) Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.
| [reply] [Watch: Dir/Any] |
by tilly (Archbishop) on May 07, 2009 at 15:42 UTC | |
by BrowserUk (Patriarch) on May 07, 2009 at 23:40 UTC | |
by Gavin (Archbishop) on May 06, 2009 at 18:56 UTC | |
Very informative and enjoyable read, even if a lot went over my head ++ BrowserUk. I do hope you find the time to complete it, perhaps it would then make a good Meditation. | [reply] [Watch: Dir/Any] |
Re: (OT) Programming languages for multicore computers
by Jenda (Abbot) on May 04, 2009 at 14:58 UTC | |
How often do you think it can be expected that your program will have the whole of the computer to (ab)use? How often do you think the computer doesn't have several other tasks to run? IMnsHO for most applications, one core will be plenty and the multiple cores will be used by multiple processes. For most of the rest, splitting the task into a few parts and running several instances of the program will be both good enough and easiest. So no, I do not think "multicore" programming is something most developers will need to care about in the near to mid future. There is one big problem with the (even partially) automatic concurrency. If you split the task into too many too small pieces then you waste more than you gain. A perfect example would be the paralelization of something like @result = map {$_*2} @source;. The task you need to do with the items is simply too simple and inexpensive. Even if you split the (huge) @source into just a few parts and parallelized those, the overhead would slam any gain. But once the chunks get sufficiently big, they usually also get too complex to parallelize automatically. So I'm a little skeptic. P.S.: I believe the concurrent stuff has been taken out of Clean. It did use to be called "Concurrent Clean" originally, but the concurrent evaluation annotations had been removed in version 2.0. Not sure what's the current status of things. Jenda | [reply] [Watch: Dir/Any] [d/l] [select] |
by salva (Canon) on May 04, 2009 at 16:28 UTC | |
How often do you think it can be expected that your program will have the whole of the computer to (ab)use? How often do you think the computer doesn't have several other tasks to run? Programs should not bother about limiting their CPUs/CPU time usage. Actually it is quite the opposite, they should use as many (concurrent) CPU resources as possible in order to finish sooner. It is the operative system who has the responsibility to ensure that some program does not degrade the performance of any other application running concurrently by just creating too many threads or any other mean. | [reply] [Watch: Dir/Any] |
by Jenda (Abbot) on May 04, 2009 at 17:05 UTC | |
I'm not talking about limiting themselves artificially. I'm talking about bending backwards to escape the single core limitation. I do believe that most often it's not needed. That most often there's plenty other processes to use the other cores. And that if all those processes attempt to use several cores, they will all finish later than if they did not bother. Jenda | [reply] [Watch: Dir/Any] |
by Zen (Deacon) on May 04, 2009 at 15:57 UTC | |
| [reply] [Watch: Dir/Any] |
Re: (OT) Programming languages for multicore computers
by creamygoodness (Curate) on May 04, 2009 at 16:56 UTC | |
Concurrency is a bug-spawning PITA. Managing access to shared resources is hard, and hard to test well. Any programming discipline which delegates responsibility for managing a large pool of shared objects to the user is a dead end. The easiest way to achieve reliable concurrency is to minimize the number of shared resources. Using OS processes as your concurrency model achieves this without imposing any constraints on choice of programming language. Unfortunately for those of us trying to write cross-platform compatible open source code, the menu of IPC techniques that work accross a wide breadth of operating systems is very limited. IMO, this is a problem which needs to be solved through OS innovation, not language innovation. | [reply] [Watch: Dir/Any] |
by samtregar (Abbot) on May 04, 2009 at 17:58 UTC | |
I think Erlang is about as close to "OS innovation" as you're likely to see in this space. The Erlang runtime is practically a little OS unto itself with its own process model, networking, storage, etc. -sam | [reply] [Watch: Dir/Any] |
Re: (OT) Programming languages for multicore computers
by lima1 (Curate) on May 11, 2009 at 09:12 UTC | |
I have to say in advance that I've never did some serious Haskell programming, but I don't think this statement is true. C is still the language of choice for this kind of tools. If you are capable of writing a high performance string matching tool, it will be quite trivial for you to make it multi-threaded. The guys in our department worked really hard for over a year on a very efficient suffix array tool, parallelizing it was a matter of weeks. Another reason to choose "high performance programming languages" when it comes to high performance computing is simply memory. I know RAM is cheep these days, but suffix arrays are demanding. For example if you want to search in the complete human genome, you'll need far more than 8gigs RAM with the most efficient suffix array implementations available. | [reply] [Watch: Dir/Any] |
by citromatik (Curate) on May 11, 2009 at 10:04 UTC | |
C is still the language of choice for this kind of tools. If you are capable of writing a high performance string matching tool ... I know that, and that is why I asked that question. Is it worth to spend >1 year coding a "very efficient suffix array tool" when (maybe) you can code it in less time being less efficient but in a language that supports parallel and concurrent computation (ideally) easily? The point about memory is relevant, but I don't know if critical in this case (I'm not trying to build suffix arrays over strings of 3Gbs) Thanks for your reply,citromatik | [reply] [Watch: Dir/Any] |
by lima1 (Curate) on May 11, 2009 at 10:46 UTC | |
You could also see it the other way: if you use C and maybe OpenMP, then the simple to implement, naive algorithms might be already fast enough. | [reply] [Watch: Dir/Any] |