Gain insights into using HPC systems and supercomputers to solve large computational problems. Explore HPC components, software stacks, job schedulers, and parallel programming, including Open MP, MPI, and GPU coding.

Learn to use High Performance Computing (HPC) Systems  and solve large computational problems.  This course assumes basic familiarity with the Bash command line environment found on GNU/Linux and other Unix-like environments. This course is of the first of its kind, should be your second step from my previous Educative course "Learn to Analyze Text Data in Bash Shell and Linux"

You'll learn:
- Intro to HPC Systems and Supercomputers
- HPC system's basic components
- HPC software stack
- HPC job schedulers and batch systems (PBS and Slurm with demos)
- Introduction to parallel programming (Open MP, MPI and GPU coding)

Video demos are included, I think it is a must have course for students, researchers, big data analysts, and developers. It will help you to utilize High Performance Computing systems at your institution. Non-interactive course. More details at:  www.scientificprogramming.io/learn-hpc

Ajay Shenoy

This course demystifies HPC and introduces you to the essential tools required to schedule and run parallel programs. Easy to understand.

India

Learn to Use HPC Systems and Supercomputers

In this lesson, we will show you a basic **MPI hello world** code and also discuss how to run an MPI program. The lesson will cover the basics of initializing MPI and running an MPI job across several processes. 

#include <mpi.h>
#include <stdio.h>

int main(int argc, char** argv) {
    // Initialize the MPI environment
    MPI_Init(NULL, NULL);

    // Get the number of processes
    int world_size;
    MPI_Comm_size(MPI_COMM_WORLD, &world_size);

    // Get the rank of the process
    int world_rank;
    MPI_Comm_rank(MPI_COMM_WORLD, &world_rank);

    // Get the name of the processor
    char processor_name[MPI_MAX_PROCESSOR_NAME];
    int name_len;
    MPI_Get_processor_name(processor_name, &name_len);

    // Print off a hello world message
    printf("Hello world from processor %s, rank %d"
           " out of %d processors\n",
           processor_name, world_rank, world_size);

    // Finalize the MPI environment.
    MPI_Finalize();
}

Let's get introduced with the functions and environment veriables used in the code:

+ `MPI_Init`, all of MPI's global and internal variables are constructed.
+ `MPI_Comm_size` returns the size of a communicator and the built-in `MPI_COMM_WORLD`  encloses all of the processes in the job, so this call should return the amount of processes that were requested for the job.
+ `MPI_Comm_rank` returns the rank of a process in a communicator. Each process inside of a communicator is assigned an incremental rank starting from zero. 
+ `MPI_Finalize` is used to clean up the MPI environment. No more MPI calls can be made after this one.

**Running the MPI hello world:** Let's create a `makefile` to compile the above code. Note that the `makefile` looks for the `MPICC` environment variable and the `mpicc` program in your installation is really just a wrapper around `gcc`, and it makes compiling and linking all of the necessary MPI routines much easier!

Supercomputers and HPC clusters

Components of a HPC system

HPC software stack

PBS - Portable Batch System

SLURM -Workload Manager

Parallel programming - OpenMP

Parallel programming - MPI

Parallel programming - GPU and CUDA

Conclusions and references

MPI - hello world!