Current Visitors:


MAMR: High-performance mapreduce programming model for material cloud applications

Weipeng Jing, Danyu Tong, Yangang Wang, Jingyuan Wang, Yaqiu Liu and Peng Zhao

Computer Physics Communications, vol. 211, pp. 79–87, 2017. Download

With the increasing data size in materials science, existing programming models no longer satisfy the application requirements. MapReduce is a programming model that enables the easy development of scalable parallel applications to process big data on cloud computing systems. However, this model does not directly support the processing of multiple related data, and the processing performance does not reflect the advantages of cloud computing. To enhance the capability of workflow applications in material data processing, we defined a programming model for material cloud applications that supports multiple different Map and Reduce functions running concurrently based on hybrid share-memory BSP called MaMR. An optimized data sharing strategy to supply the shared data to the different Map and Reduce stages was also designed. We added a new merge phase to MapReduce that can efficiently merge data from the map and reduce modules. Experiments showed that the model and framework present effective performance improvements compared to previous work.

Architecture of the MaMR framework
Architecture of the MaMR framework

If you find our work is helpful for your research, please kindly consider citing our paper.

 

@article{jing2017mamr,

  title={MaMR: High-performance MapReduce programming model for material cloud applications},

  author={Jing, Weipeng and Tong, Danyu and Wang, Yangang and Wang, Jingyuan and Liu, Yaqiu and Zhao, Peng},

  journal={Computer Physics Communications},

  volume={211},

  pages={79--87},

  year={2017},

  publisher={Elsevier}

}