Towards the availability of Java bindings in open MPI

Vega-Gisbert, Oscar; Román, José E.; Groß, Siegmar; Squyres, Jeffrey M.

doi:10.1145/2488551.2488599

Cited by 9 publications

(4 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The DA-MDS implementation in SPIDAL Java, for example, has two call sites to MPI allgatherv collective, BC-Comm and MMComm, written using OpenMPI Java binding [23]. They both communicate an identical number of data elements, except one routine is called more times than the other.…”

Section: Technical Evaluationmentioning

confidence: 99%

SPIDAL Java: High Performance Data Analytics with Java and MPI on Large Multicore HPC Clusters

Ekanayake

Kamburugamuve

Fox

2016

24th High Performance Computing Symposium

View full text Add to dashboard Cite

Within the last few years, there have been significant contributions to Java-based big data frameworks and libraries such as Apache Hadoop, Spark, and Storm. While these systems are rich in interoperability and features, developing high performance big data analytic applications is challenging. Also, the study of performance characteristics and high performance optimizations is lacking in the literature for these applications. By contrast, these features are well documented in the High Performance Computing (HPC) domain and some of the techniques have potential performance benefits in the big data domain as well. This paper presents the implementation of a high performance big data analytics library-SPIDAL Java-with a comprehensive discussion on five performance challenges, solutions, and speedup results. SPIDAL Java captures a class of global machine learning applications with significant computation and communication that can serve as a yardstick in studying performance bottlenecks with Java big data analytics. The five challenges present here are the cost of intra-node messaging, inefficient cache utilization, performance costs with threads, overhead of garbage collection, and the costs of heap allocated objects. SPIDAL Java presents its solutions to these and demonstrates significant performance gains and scalability when running on up to 3072 cores in one of the latest Intel Haswell-based multicore clusters.

show abstract

Section: Technical Evaluationmentioning

confidence: 99%

SPIDAL Java: High Performance Data Analytics with Java and MPI on Large Multicore HPC Clusters

Ekanayake

Kamburugamuve

Fox

2016

24th High Performance Computing Symposium

View full text Add to dashboard Cite

show abstract

“…In order to facilitate writing parallel Java code for shared memory systems, Java is equipped with a feature-rich threading API. In order to teach programming distributed memory systems, we had the option to choose between various Java MPI libraries including MPJ Express [5], FastMPJ [7], and Open MPI Java [8]. In this context, we choose MPJ Express that is being developed and maintained at NUST Pakistan.…”

Section: Motivation For Using Javamentioning

confidence: 99%

Teaching Parallel Programming Using Java

Shafi

Akhtar

Javed

et al. 2014

2014 Workshop on Education for High Performance Computing

View full text Add to dashboard Cite

This paper presents an overview of the "Applied Parallel Computing" course taught to final year Software Engineering undergraduate students in Spring 2014 at NUST, Pakistan. The main objective of the course was to introduce practical parallel programming tools and techniques for shared and distributed memory concurrent systems. A unique aspect of the course was that Java was used as the principle programming language. The course was divided into three sections. The first section covered parallel programming techniques for shared memory systems that include multicore and Symmetric Multi-Processor (SMP) systems. In this section, Java threads was taught as a viable programming API for such systems. The second section was dedicated to parallel programming tools meant for distributed memory systems including clusters and network of computers. We used MPJ Express-a Java MPI library-for conducting programming assignments and lab work for this section. The third and the final section covered advanced topics including the MapReduce programming model using Hadoop and the General Purpose Computing on Graphics Processing Units (GPGPU).

show abstract

“…The use of JNI ensures the minimum added overhead to the performance of native MPI functions. The maintenance complexity pertained to the first-generation MPI bindings for Java, as discussed by [1], is minimized by a tight integration with the native build system of the MPI library, as presented by [10]. In terms of the functional possibilities, MPI stays ahead of all other alternatives.…”

Section: Pros and Cons Analysis For Use With Hpcsmentioning

confidence: 99%

HPC in Big Data Age

Cheptsov

2014

Proceedings of the 21st European MPI Users' Group Meeting

View full text Add to dashboard Cite

The current IT technologies have a strong need for scaling up the high-performance analysis to large-scale datasets. Tremendously increased over the last few years volume and complexity of data gathered in both public (such as on the web) and enterprise (e.g. digitalized internal document base) domains have posed new challenges to providers of high performance computing (HPC) infrastructures, which is recognised in the community as Big Data problem. On contrast to the typical HPC applications, the Big Data ones are not oriented on reaching the peak performance of the infrastructure and thus offer more opportunities for the "capacity" infrastructure model rather than for the "capability" one, making the use of Cloud infrastructures preferable over the HPC. However, considering the more and more vanishing difference between these two infrastructure types, i.e. Cloud and HPC, it makes a lot of sense to investigate the abilities of traditional HPC infrastructure to execute Big Data applications as well, despite their relatively poor efficiency as compared with the traditional, very optimized HPC ones. This paper discusses the main state-of-the-art parallelisation techniques utilised in both Cloud and HPC domains and evaluates them on an exemplary text processing application on a testbed HPC cluster.

show abstract

Towards the availability of Java bindings in open MPI

Cited by 9 publications

References 5 publications

SPIDAL Java: High Performance Data Analytics with Java and MPI on Large Multicore HPC Clusters

SPIDAL Java: High Performance Data Analytics with Java and MPI on Large Multicore HPC Clusters

Teaching Parallel Programming Using Java

HPC in Big Data Age

Contact Info

Product

Resources

About