Opening Session

Self-presentation: 1 minute per person

Lecture 1 Description of theme / lecture

Practice 1 Description about Practice 1

Practice 2 Description about Practice 1

Writing parallel software

• Sebastien Ponce (CERN)

Parallel and Optimised Scientific Software - lecture 1 - Amdahl's and Gustafson's laws - Asynchronous execution - Finding concurrency, task vs. data parallelism - Using threading in C++ and Python, comparison with multi-process - Resource protection and thread safety - Locks, thread local storage, atomic operations

Writing efficient software

• Sebastien Ponce (CERN)

Parallel and Optimised Scientific Software - lecture 2 - Why Python and C++ ? - Recent evolutions: C++ 11/14/17 - Modern features of C++ related to performance - Templating versus inheritance, pros and cons of virtual inheritance - Python 3, and switching from Python 2

Announcements

Low-level Performance Optimization

• Guilherme Amadio (CERN)

CPU Architecture and High Performance - lecture 3 - Main sources of performance bottlenecks - Floating point arithmetics performance - Advanced low-level performance tuning

Data-Oriented Design

• Guilherme Amadio (CERN)

CPU Architecture and High Performance - lecture 4 - Principles of data-oriented design - Memory access and data-type profiling - Data structure performance optimization

Optimizing existing large codebase

• Sebastien Ponce (CERN)

Parallel and Optimised Scientific Software - lecture 3 - Measuring performance, tools and key indicators - Improving memory handling - The nightmare of thread safety - Code modernization and low level optimizations - Data structures for efficient computation in modern C++

Student lightning talks

CPU Architecture and High Performance - exercise 2

• Guilherme Amadio (CERN)

Scientific computing on heterogeneous architectures

• Daniel Hugo Campora Perez (NVIDIA)

Programming for Heterogeneous Architectures - lecture 1 - Introduction to heterogeneous architectures and the performance challenge - From general to specialized: Hardware accelerators and applications - Type of workloads ideal for different accelerators - Trade-offs between multi-core and many-core architectures - Implications of heterogeneous hardware on the design and architecture of scientific software - Embarrassingly parallel scientific applications in HPC and CERN

Optimizing existing large codebase - exercise 1

• Sebastien Ponce (CERN)

Optimisation of an existing, production grade large codebase

Announcements

Optimizing existing large codebase - exercise 2

• Sebastien Ponce (CERN)

Optimisation of an existing, production grade large codebase

River rafting excursion - Cetina River

Transport to Medils

Performant programming for GPUs

• Daniel Campora (NVIDIA)

Programming for Heterogeneous Architectures - lecture 2 - From SIMD to SPMD, a programming model transition - Thread and memory organization - Basic building blocks of a GPU program - Control flow, synchronization, atomics

Performant programming for GPUs

• Daniel Campora (NVIDIA)

Programming for Heterogeneous Architectures - lecture 3 - Data locality, coalesced memory accesses, tiled data processing - GPU streams, pipelined memory transfers - Under the hood: branchless, warps, masked execution - Debugging and profiling a GPU application

Announcements

Practical vectorization

• Sebastien Ponce (CERN)

Parallel and Optimised Scientific Software - lecture 4 - Measuring vectorization level - What to expect from vectorization - Preparing code for vectorization - Vectorizing techniques in C++: intrinsics, libraries, autovectorization

Practical vectorization - exercise

• Sebastien Ponce (CERN)

Programming for heterogeneous architectures - exercise

• Daniel Campora (NVIDIA)

Programming for heterogeneous architectures - exercise

• Daniel Campora (NVIDIA)

The Future of the Universe and Humanity

• Ivica Puljak (Technical University of Split FESB)

Debrief of exercises

Design patterns and best practices

• Daniel Campora (NVIDIA)

Programming for Heterogeneous Architectures - lecture 4 - Good practices: single precision, floating point rounding, avoid register spilling, prefer single source - Other standards: SYCL, HIP, OpenCL - Middleware libraries and cross-architecture compatibility - Reusable parallel design patterns with real-life applications

Announcements

Programming for heterogeneous architectures - exercise

• Daniel Campora (NVIDIA)

Programming for heterogeneous architectures - exercise

• Daniel Campora (NVIDIA)

Exam

Closing Session

Departures