Introduction to HPC & the Athena Supercomputer

• Leszek Grzanka
• Klemens Noga (ACC Cyfronet AGH)

A brief overview of High-Performance Computing (HPC) concepts, logging into the Athena supercomputer at ACK Cyfronet AGH, and the basics of accessing computational resources.

Environment Setup & Python Memory Model

• Leszek Grzanka

Initializing the JupyterHub environment, verifying dependencies, and exploring the performance differences between standard Python lists and NumPy arrays

Introduction to Data Analysis with Pandas

• Leszek Grzanka

Hands-on introduction to Pandas using real-world weather datasets. Covers basic data manipulation, filtering, and preparation for parallel processing.

Scaling Up: First Steps with Dask

• Leszek Grzanka

Transitioning from Pandas to Dask DataFrames. An introduction to distributed data processing and executing our first operations on larger-than-memory datasets.

Dask Performance & Lazy Evaluation

• Leszek Grzanka

A condensed overview of how Dask works under the hood. Understanding task graphs, lazy evaluation mechanics, and benchmarking Pandas versus Dask performance.

Processing Scientific Data at Scale

• Leszek Grzanka

Analyzing real high-frequency physics datasets (Parquet format) from proton therapy beam measurements using single-node Dask clusters.

Multi-node Computing & Final Challenge

• Leszek Grzanka

Utilizing dask-jobqueue and SLURMCluster to scale out computations across multiple Athena nodes. Plenty of hands-on time to tackle the open-ended multi-dataset challenge.