Programming for multi-core performance - in Java?

Performance is a huge concern when writing any Java code, especially since many still do not realize the benefits of coding to a hotspot vm and believe Java to be a "slower" language than something like C/C++. As someone with a hardware background, I frequently hear that faster computers are the answer. To an extent, they are correct. To this end, processor architects have been putting more processing power in our beloved machines by adding nifty new technology like hyper-threading, cores, and increasing the on-die caches in recent years. All this is great; however, we can tap this power to a greater extent if we software guys have ways to program closer to this type of hardware.

Back in college, we had a homework assignment to finish adding the hardware in a CPU necessary for the jump assembly command. We did our diagrams and loaded it all into our CAD tool. After we had the full layout of the processor in memory (first time of course!) we had to do some testing. So, we used VERILOG for the first time, and doing so was quite an experience. We were rudely introduced to a programming style that was purely parallel; we were basically coding electrical impulses through tiny wires. In any case, it is a very different feel than doing your standard sequential programming (Java/C/C++/C#)and it is more than a little difficult to get accustomed to. The first few times we tried it we encountered many race-through issues and problems making sure all the code paths lined up. It was quite a learning experience indeed.

For the past few years, there have been many discussions on how to best program threads for this environment in the C/C++ languages. Intel has many good documents on the OpenMP API (here's a good one), along with some interesting comparisons (and arguments) to using PThreads as well. However, what about us Java guys, and could there be a better way than straight-up thread programming?

As I said before, parallel programming is interesting, but it's extremely difficult to master. In addition, we must take into account scalability - programming for 2 cores may be different than programming for 100 cores (thinking ahead, of course). Is it unreasonable for us to find a way to abstract the threading and split up work when the JIT compiling takes place? I hope not, because doing so would be really nifty, especially since the Jedi Thread Masters among us are few and far between.

Enter JSR-166 and JSR-166y fork/join framework! According to Doug Lea, the framework is "a style of parallel programming in which problems are solved by (recursively) splitting them into subtasks that are solved in parallel, waiting for them to complete, and then composing results." Here is his pseudo code to demonstrate:

[cc lang="xquery"]Result solve(Problem problem) { if (problem is small) directly solve problem else { split problem into independent parts fork new subtasks to solve each part join all subtasks compose result from subresults } } [/cc]

I think this is a really big step in the right direction for Java, but is not quite the holy grail of multi-thread abstraction for the masses. I'm still researching this framework, but at first glance, the framework appears to allow the application to be coded such that depending on the size of the task/algorithm, one can easily fork/spawn a separate subtask that is computed in parallel (waiting for them all to complete of course), then joining them back together. This has enormous potential in my opinion!

I'm going to continue my research and will post my thoughts here. In the meantime, take a look at Doug Lea's paper, "A Java Fork/Join Framework" as well as some of the existing JSR-166 collections in Java SE 6: Deque, BlockingDeque, NavigableSet,NavigableMap, ConcurrentNavigableMap, ArrayDeque, LinkedBlockingDeque, ConcurrentSkipListSet,ConcurrentSkipListMap, TreeMap.

Resources:

A Java Fork/Join Framework by Doug Lea

Concurrency Interest Site

JSR-166 Site