Using machine learning frameworks on LUMI

• Maciej Szpindler

This part shows how to setup environment for AI workflows on a real supercomputing system. It combines common HPC tools with popular machine learning frameworks relying on GPU offloading and multiprocessing across multiple computing nodes.

Working with large language models using Megatron-DeepSpeed on LUMI

• Maciej Szpindler

This builds up on details from the first part to setup specific environment for LLM processing. It uses Megatron-DeepSpeed framework to experiment with pretrained data for prompt engineering tasks.