README.md

Uni.lu High Performance Computing (HPC) Tutorials

 Copyright (c) 2013-2021 UL HPC Team <[email protected]>

This repository holds a set of tutorials to help the users of the UL HPC platform to better understand or simply use our platform.

This is the main page of the documentation for this repository, which relies on MkDocs and the Read the Docs theme. In particular, the latest version of these tutorials is available online:

http://ulhpc-tutorials.rtfd.io

The list of proposed tutorials is continuously evolving. They used on a regular basis during the UL HPC School we organise.

So far, the following tutorials are proposed:

Category	Description	Level
	Pre-requisites and Setup instructions	beginners
Basic	Introduction to UNIX/Linux Shell and Command lines	beginners
	Getting Started on the UL HPC platform	beginners
	Monitoring & Profiling I: why, what, how, where to look	beginners
Emb. Parallel Jobs	GNU Parallel	beginners
	HPC Management of Sequential and Embarrassingly Parallel Jobs	beginners
Scheduling	Advanced scheduling with SLURM	intermediate
Software Management	HPC Software Building: optimizing and complementing the ULHPC software set	beginners
	The Spack package manager for supercomputers	beginners
Data Management	Data Management on UL HPC Facility	beginners
Debuging & Profiling	Know Your Bugs: Weapons for Efficient Debugging	intermediate
	Advanced debugging on the UL HPC platform	intermediate
	(OLD) Unified profiling and debugging with Allinea	intermediate
	(OLD) Direct, Reverse and parallel Memory debugging with TotalView	intermediate
MPI	Scalable Science and Parallel computations with OpenMP/MPI	intermediate
	OSU Micro-Benchmarks	intermediate
	High-Performance Linpack (HPL) benchmarking on UL HPC platform	intermediate
	HPCG benchmarking on UL HPC platform	intermediate
Python	Prototyping with Python	beginners
	Python: Use Jupyter notebook on UL HPC	intermediate
	Use Python Celery on UL HPC	advanced
	Scalable computing with Dask	advanced
	Parallel machine learning with scikit-learn	intermediate
	Parallel evolutionary computing with Scoop/Deap	intermediate
Mathematics	MATLAB (interactive, passive and sequential jobs)	intermediate
	Advanced MATLAB execution: checkpointing and parallel jobs	advanced
	R / Statictical Computing	intermediate
	Cplex / Gurobi	intermediate
Bioinformatics	Bioinformatics software on the UL HPC platform	intermediate
	Galaxy Introduction Exercise: From Peaks to Genes	intermediate
CFD/MD/Chemistry	Scalable Science II (Advanced - Computational Physics, Chemistry & Engineering apps)	advanced
Big Data	Running Big Data Application using Apache Hadoop and Spark	intermediate
Machine/Deep Learning	Machine and Deep learning workflows	intermediate
	Distributed Deep Learning with Horovod	intermediate
	Distributed Large Language Model with HuggingFace and DeepSpeed	advanced
Containers	Singularity	advanced
	Singularity with Infiniband	advanced
	Reproducibility and HPC Containers	advanced
Virtualization	(OLD) Create and reproduce work environments using Vagrant	intermediate
	Deploying virtual machines with Vm5k on Grid'5000	intermediate
GPU Programming	Introduction to GPU programming with CUDA (C/C++)	intermediate
	Image Convolution with GPU and CUDA	advanced
	Introduction to OpenCL	intermediate
	Introduction to OpenACC Programming Model (C/C++ and Fortran)	intermediate
	Solving the Laplace Equation on GPU with OpenAcc	advanced
IPU Programming	Tensorflow on Graphcore IPU	intermediate
Misc	Reproducible Research at the Cloud Era	intermediate

List of contributors

• See docs/contacts.md.
• In the advent where you want to contribute yourself to these tutorials, do not hesitate! See below for instructions.

Issues / Feature request

• You can submit bug / issues / feature request using the ULHPC/tutorials Tracker.
  • See also docs/contributing/ for further information.

__ Contributing__

• If you want to contribute to the code, you shall be aware of the way this module is organized.
• These elements are detailed on docs/contributing.md.
  • You are more than welcome to contribute to its development by sending a pull request.

Online Documentation

• Read the Docs aka RTFD hosts documentation for the open source community and the ULHPC/sysadmins has its documentation (see the docs/ directly) hosted on readthedocs.
• See docs/rtfd.md for more details.

Licence

This project and the sources proposed within this repository are released under the terms of the GPL-3.0 licence.