Parallel and High Performance Computing

Parallel and High Performance Computing offers techniques guaranteed to boost your code?s effectiveness.SummaryComplex calculations, like training deep learning models or running largescale simulations, can take an extremely long time. Efficient parallel programming can save hoursor even daysof computing time. Parallel and High Performance Computing shows you how to deliver faster runtimes, greater scalability, and increased energy efficiency to your programs by mastering parallel techniques for multicore processor and GPU hardware.About the technologyWrite fast, powerful, energy efficient programs that scale to tackle huge volumes of data. Using parallel programming, your code spreads data processing tasks across multiple CPUs for radically better performance. With a little help, you can create software that maximizes both speed and efficiency.About the bookParallel and High Performance Computing offers techniques guaranteed to boost your code?s effectiveness. You?ll learn to evaluate hardware architectures and work with industry standard tools such as OpenMP and MPI. You?ll master the data structures and algorithms best suited for high performance computing and learn techniques that save energy on handheld devices. You?ll even run a massive tsunami simulation across a bank of GPUs.What s insidePlanning a new parallel projectUnderstanding differences in CPU and GPU architectureAddressing underperforming kernels and loopsManaging applications with batch schedulingAbout the readerFor experienced programmers proficient with a highperformance computing language like C, C++, or Fortran.About the authorRobert Robey works at Los Alamos National Laboratory and has been active in the field of parallel computing for over 30 years. Yuliana Zamora is currently a PhD student and Siebel Scholar at the University of Chicago, and has lectured on programming modern hardware at numerous national conferences.Table of ContentsPART 1 INTRODUCTION TO PARALLEL COMPUTING1 Why parallel computing?2 Planning for parallelization3 Performance limits and profiling4 Data design and performance models5 Parallel algorithms and patternsPART 2 CPU: THE PARALLEL WORKHORSE6 Vectorization: FLOPs for free7 OpenMP that performs8 MPI: The parallel backbonePART 3 GPUS: BUILT TO ACCELERATE9 GPU architectures and concepts10 GPU programming model11 Directivebased GPU programming12 GPU languages: Getting down to basics13 GPU profiling and toolsPART 4 HIGH PERFORMANCE COMPUTING ECOSYSTEMS14 Affinity: Truce with the kernel15 Batch schedulers: Bringing order to chaos16 File operations for a parallel world17 Tools and resources for better code