Understanding Performance Bottlenecks and Reliability Issues in Python Code

Presenters: Aleksandr Lykhin

This workshop focuses on practical techniques for testing, debugging, and profiling Python applications in an HPC environment, with an emphasis on identifying real performance bottlenecks and hotspots. A scientific code developed rapidly and collaboratively often works on small test cases but reveals performance, scalability, or memory issues when executed at scale or execution failures when subjected to full-size data, parallel execution, or production-like workflows. Participants will learn how to systematically diagnose slowdowns, excessive memory usage, and parallelization issues using modern profiling and debugging tools, and how to troubleshoot common failures encountered during development. The session emphasizes hands-on workflows commonly used on shared clusters, such as Midway3. 

Learning Objectives

By the end of this workshop, participants will be able to:

• Apply testing frameworks to improve code stability and accelerate prototype development
• Use debugging tools to isolate issues and diagnose prototype failures
• Leverage profiling tools to identify CPU, memory, and I/O bottlenecks in Python applications running on HPC systems

Level: Intermediate

Prerequisites: Basic familiarity with Python, VSCode, and running jobs on an HPC cluster