LG259: Job parallelization using distributed network (MPI) programming on HPC.

Friday, February 19, 2016 - 12:00
Jhimli Mitra, PhD
In this session will quickly walk through the directives on how to organize your data and programs efficiently on HPC to reduce latency issues in accessing data. The primary focus of this session would be to show how multiple jobs can be parallelized on distributed computers on HPC with optimal use of resources. Typically, this session will benefit those who still process the same instruction on multiple data serially. For example, if you have a for-loop running bias correction for 100 data in your local machine each taking 1 minute, totaling to 100 minutes. You can efficiently parallelize this as 100 jobs in 100 different machines, effectively returning all your outputs in 1 minute approximately (depending if resources are available).