11–13 Dec 2024
CERN
Europe/Zurich timezone

Development and implementation of an environmentally friendly Workload Management System in DiracX

12 Dec 2024, 11:30
15m
31/3-004 - IT Amphitheatre (CERN)

31/3-004 - IT Amphitheatre

CERN

105
Show room on map

Speaker

Henryk Giemza (National Centre for Nuclear Research (PL))

Description

The goal of this thesis is to develop an optimized task allocation algorithm for the Workload Management System (WMS) in DiracX, aimed at reducing energy consumption and maximizing the utilization of renewable energy sources. To achieve this, a combination of advanced techniques—including numerical optimization, clustering, and artificial intelligence (e.g., reinforcement learning)—will be explored. The proposed algorithm will take into account a variety of factors, such as (but not limited to):
* The energy mix at the country level (including forecasts, if available),
* Local resource center optimizations (e.g., waste heat recovery, cooling methods, dynamic downclocking),
* Global and local policy considerations (e.g., resource and climate pledges, energy regulations),
* The efficiency of high-energy physics (HEP) jobs at resource centers (based on factors like hardware performance, job failure rates, outages, etc.).

As this project is still in its early stages, the specific algorithm and the results of subsequent simulations and real-world tests have not yet been developed. However, this presentation will outline potential strategies for integrating environmental sustainability aspects into job allocation mechanisms on the grid and discuss methods for validating the effectiveness of the proposed solutions.

Primary author

Henryk Giemza (National Centre for Nuclear Research (PL))

Presentation materials