19–25 Oct 2024
Europe/Zurich timezone

Implementation and Performance Analysis of the ALICE grid middleware JAliEn’s Job Optimizer

23 Oct 2024, 16:33
18m
Room 2.B (Conference Room)

Room 2.B (Conference Room)

Talk Track 4 - Distributed Computing Parallel (Track 4)

Speaker

Haakon Andre Reme-Ness (Western Norway University of Applied Sciences (NO))

Description

This paper presents a comprehensive analysis of the implementation and performance enhancements of the new job optimizer service within the JAliEn (Java ALICE environment) middleware framework developed for the ALICE grid. The job optimizer service aims to efficiently split large-scale computational tasks into smaller grid jobs, thereby optimizing resource utilization and throughput of the grid by ensuring more grid resources are able to match with grid jobs. New functionalities for users of the grid are described, while also delving into back-end changes that have improved the job optimizer service.

Through testing and evaluation in a production environment, significant improvements in database performance, faster job splitting, and better scalability have been observed when doing comparative analysis against the legacy job optimization service. Further potential improvements in the future will also be explored.

This paper will also provide a look into the technical intricacies of the new job optimizer service, highlighting functionalities, implementation strategies, and integration within the existing JAliEn framework. Furthermore, insights into the lessons learned and challenges encountered during the implementation phase, deployment, and operationalization of the job optimizer service will be discussed.

Primary authors

Dr Bjarte Kileng (Western Norway University of Applied Sciences (NO)) Costin Grigoras (CERN) Haakon Andre Reme-Ness (Western Norway University of Applied Sciences (NO)) Haavard Helstrup (Western Norway University of Applied Sciences (NO)) Latchezar Betev (CERN) Maksim Melnik Storetvedt (CERN)

Presentation materials