The Design of Advanced Communication to Reduce Memory Usage for Exa-scale Systems

Enhancing in-memory efficiency for MapReduce-based data processing

“…Memory efficiency has been the subject of study of many previous works. Some of them [10,11] enhanced the scalability of the Message Passing Interface (MPI) in terms of memory usage. Although these works are not focused on Big Data processing, some of the challenges they face are similar to our case (e.g., keeping memory scalability as the number of processes grows).…”