Let's project that over the next 30 years. In 2010 we'll have 8-cores. If development continues in 2015 we'll have 32-cores. In 2020 we'll have 256-cores. In 2025 we'll have 1K cores. In 2030 we'll have 4K cores. In 2035 we'll have 16K cores. And in 2040 we'll have 64K cores. That's based on 5 year quadrupling time, which is below the currently measured rate.
Based on what I know of operating systems, there will be no obvious problems scaling operating systems in the next 5 years - Windows and Linux both scale to 32 CPUs fairly well. Operating systems will appear to scale for the next 5 years, albeit with diminishing returns. The real challenges come in the following 5 years. Given how slow our profession is to notice the obvious, it may take 20 years for it to become widely understood that massively multi-threaded systems don't scale that well. And when that happens I'll be able to say, "I knew that ages ago!"
Now let's discuss my "straw man". When I was talking about scalability limits, I was not talking about the scalability limits of the algorithms, but instead about the scalability limits of conventional operating systems. Which needs to synchronize CPUs for various reasons, such as sorting out the routing of I/O. Both between processes and with the outside world.
Moving forward, you raise an excellent point about the architecture of the Cell processor. I am not familiar with that architecture, but looking at it you do effectively get NUMA while hiding the details from the operating system.
However straightforwardly multi-threaded programs are not going to be magically exempt from scalability issues due to the overhead of memory synchronization. When 2 threads that are remote from each other both access a piece of memory which needs to look consistent between them, there has to be locking under the hood at some level to make that happen. With traditional multi-threaded programs, there are no guarantees on any part of memory being shared, and this leads to complications. With pipelined data as in your previous suggestion, you've controlled what sections of memory need to be locked, and how that locking needs to happen.
About the only guarantee for the future is that the world will be interesting as people figure all of this out.
Update: I fixed my projections into the future. I also said when I thought people would notice these issues.
In reply to Re^4: (OT) Programming languages for multicore computers
by tilly
in thread (OT) Programming languages for multicore computers
by citromatik
| For: | Use: | ||
| & | & | ||
| < | < | ||
| > | > | ||
| [ | [ | ||
| ] | ] |