2012-02-19 @ admin
Message-passing is the most widely used parallel programming paradigm as it is
highly portable, scalable and usually provides good performance. It is the preferred
choice for parallel programming distributed memory systems such as clusters, which
can provide higher computational power than shared memory systems. Regarding the
languages compiled to native code (e.g., C and Fortran),MPI is the standard interface
for message-passing libraries.
With respect to Java, there have been several implementations of Java messagepassing
libraries since its introduction. Although initially each project developed
their own MPI-like binding for the Java language, current projects generally adhere
to one of the two main proposed APIs, the mpiJava 1.2 API, which supports an
MPI C++-like interface for the MPI 1.1 subset, and the JGF MPJ API, which
is the proposal of the Java Grande Forum (JGF) to standardize the MPI-like Java
API. The collective communication primitives are essential part of the different MPJ
APIs, both in terms of number of methods and widespread use.
MPJ libraries can be implemented in two ways: (1) wrapping an underlying native
messaging library like MPI through Java Native Interface (JNI); or (2) using a
“pure” Java (100% Java) approach, based on RMI or sockets. Each solution fits with
specific situations, but presents associated trade-offs. The use of the pure Java approach
ensures portability, but it might not be the most efficient solution, especially
130 G.L. Taboada et al.
in the presence of high-speed communication hardware and when using RMI or Java
Message Service (JMS) as these technologies are oriented to distributed computing
on loose coupled peers and show high start-up latencies. The use of JNI has portability
problems, although usually in exchange for higher performance. With respect
to the MPJ collective library implementation, in a wrapper MPJ library it consists
of a collection of wrapper classes that rely on an underlying MPI collective library
implementation, whereas a pure Java MPJ library requires a whole collectives implementation,
usually on top of point-to-point communications. Although the MPJ
proposals do not discuss the handling of collective libraries, it is possible to integrate
third-party collective libraries in any MPJ project, especially if the collective
operations are based on MPJ point-to-point operations.
The mpiJava library is a wrapper implementation which provides efficient
communication resorting to an underlying native MPI library, adding a reduced JNI
overhead. However, despite its usually high performance, mpiJava currently only supports
some native MPI implementations, as wrapping a wide number of functions,
especially the collectives, and heterogeneous runtime environments entails an important
maintaining effort. Additionally, this implementation presents instability problems,
derived from the native code wrapping, and it is not thread-safe, being unable
to take advantage of multi-core systems through multithreading.
As a result of these drawbacks, the mpiJava project maintenance has been superseded
by the development of MPJ Express, a “pure” Java message-passing
implementation of the mpiJava 1.2 API specification. MPJ Express is thread-safe and
presents a modular design which includes a pluggable architecture of communication
devices that allows to combine the portability of the “pure” Java New I/O package
(Java NIO) communications (niodev device) with the high performance Myrinet support
(through the native Myrinet eXpress—MX—communication library in mxdev
device). Furthermore, this project is the most active in terms of adoption by the HPC
community, presence on academia and production environments, and available documentation.
This project is also stable and publicly available along with their source
code at http://mpj-express.org. However, MPJ Express is currently distributed with a
poorly scalable collective library that is a limiting factor for its definitive adoption in
HPC.
Additional MPJ implementations, such as MPJ/Ibis, include their own collective
library, although quite often these implementations are incomplete and their
primitives implement poorly scalable algorithms, as they usually do not take advantage
of non-blocking communications. The aim of this work is to provide MPJ libraries
with a portable, efficient and scalable collective library. Moreover, this library
has been integrated within MPJ Express, showing significantly higher scalability for
MPJ collective communications.
P.S. Java programming is hard work which you have to practice. If you looking for ways to make quick money it’s not for you…