[Table of Contents]

Plasma and Fusion Research

Volume 6, 2401084 (2011)

Regular Articles

HPC Parallel Programming Model for Gyrokinetic MHD Simulation
Hiroshi NAITOU, Yusuke YAMADA, Shinji TOKUDA1,2), Yasutomo ISHII2) and Masatoshi YAGI2,3)
Yamaguchi University, 2-16-1 Tokiwadai, Ube 755-8611, Japan
Research Organization for Information Science and Technology, 2-32-3 Kita-shinagawa, Shinagawa-ku, Tokyo 140-0001, Japan
Japan Atomic Energy Agency, 801-1 Mukoyama, Naka 311-0193, Japan
Kyushu University, 6-1 Kasuga, Fukuoka 816-8580, Japan
(Received 22 December 2010 / Accepted 7 February 2011 / Published 12 July 2011)


The 3-dimensional gyrokinetic PIC (particle-in-cell) code for MHD simulation, Gpic-MHD, was installed on SR16000 (“Plasma Simulator”), which is a scalar cluster system consisting of 8,192 logical cores. The Gpic-MHD code advances particle and field quantities in time. In order to distribute calculations over large number of logical cores, the total simulation domain in cylindrical geometry was broken up into NDD-r × NDD-z (number of radial decomposition times number of axial decomposition) small domains including approximately the same number of particles. The axial direction was uniformly decomposed, while the radial direction was non-uniformly decomposed. NRP replicas (copies) of each decomposed domain were used (“particle decomposition”). The hybrid parallelization model of multi-threads and multi-processes was employed: threads were parallelized by the auto-parallelization and NDD-r ×NDD-z ×NRP processes were parallelized by MPI (message-passing interface). The parallelization performance of Gpic-MHD was investigated for the medium size system of Nr × Nθ × Nz = 1025 × 128 × 128 mesh with 4.196 or 8.192 billion particles. The highest speed for the fixed number of logical cores was obtained for two threads, the maximum number of NDD-z, and optimum combination of NDD-r and NRP. The observed optimum speeds demonstrated good scaling up to 8,192 logical cores.


gyrokinetic theory, particle-in-cell code, magnetohydrodynamics, tokamak, symmetric multiprocessing, message-passing interface, multi-threads, multi-processes

DOI: 10.1585/pfr.6.2401084


  • [1] W.W. Lee, J. Comput. Phys. 72, 243 (1987).
  • [2] T.S. Hahm, W.W. Lee and A. Brizard, Phys. Fluids 31, 1940 (1988).
  • [3] H. Naitou, H. Hashimoto, Y. Yamada, S. Tokuda and M. Yagi, J. Plasma Fusion Res. SERIES 8, 1158 (2009).
  • [4] H. Naitou, H. Hashimoto, Y. Yamada, S. Tokuda and M. Yagi, accepted in Progress in Nuclear Science and Technology.
  • [5] H. Naitou, K. Tsuda, W.W. Lee and R.D. Sydora, Phys. Plasmas 2, 4257 (1995).
  • [6] H. Naitou, T. Sonoda, S. Tokuda and V.K. Decyk, J. Plasma Fusion Res. 72, 259 (1996).
  • [7] I. Manuilskiy and W.W. Lee, Phys. Plasmas 7, 1381 (2000).
  • [8] W.W. Lee, J.L.V. Lewandowski, T.S. Hahm and Z. Lin, Phys. Plasmas 8, 4435 (2001).
  • [9] H. Naitou, K. Kobayashi, H. Hashimoto, T. Andachi, W.W. Lee, S. Tokuda and M. Yagi, “Proposal of a brand-new gyrokinetic algorithm for global MHD simulation”, Bull. APS, 51st Annual Meeting of the Division of Plasma Physics, Nov.2-6, 2009, Atlanta, Georgia (2009).
  • [10] H. Naitou, Y. Yamada, K. Kajiwara, W.W. Lee, S. Tokuda and M. Yagi, submitted to Plasma Science and Technology.
  • [11] H.A. van del Vorst, SIAM Journal on Scientific and Statistical Computing 13, 631 (1992).

This paper may be cited as follows:

Hiroshi NAITOU, Yusuke YAMADA, Shinji TOKUDA, Yasutomo ISHII and Masatoshi YAGI, Plasma Fusion Res. 6, 2401084 (2011).