19–25 Oct 2024
Europe/Zurich timezone

Integration of A Supercomputing Center in LHAASO Distributed Computing System

24 Oct 2024, 16:15
18m
Room 2.B (Conference Room)

Room 2.B (Conference Room)

Talk Track 4 - Distributed Computing Parallel (Track 4)

Speaker

Qingbao Hu (IHEP)

Description

LHAASO experiment is a new generation multi-component experiment designed to study cosmic rays and gamma-ray astronomy. The data volume from LHAASO are currently reaching to ~40PB and ~11PB of new data will be generated every year in the future. Such scale of data needs a big scale of computing resources to process. For LHAASO experiment, there are several types of computing sites to join the LHAASO distributed computing system, that a supercomputing center is one of them. To adding a supercomputing center, several components are developed: 1. a site job agent based on HTCondor is deployed in front of the batch system of the site; 2. two proxy services are respectively responsible to job schedule and data transfer from central side to the site across the firewall of supercomputing center; 3. a service synchronizes the necessary data to a shared filesystem of the site, including the software. 4. job running environment is wrapped inside a container. The services have been deployed in a Chengdu Supercomputing Center and the test results indicate an acceptable performance on job schedule (thousands of jobs) and data transfer (tens of TB).

Primary authors

Qingbao Hu (IHEP) Xiaowei Jiang (Chinese Academy of Sciences (CN))

Co-authors

Mr Chaoqi Guo (the Institute of High Energy Physics (IHEP) of Chinese Academy of Sciences) Jingyan Shi (Chinese Academy of Sciences (CN)) Yaodong CHENG (IHEP, Beijing) Prof. Yaodong Cheng (IHEP, CAS) Dr Yujiang BI (Institute of High Energy Physics, Chinese Academy of Sciences)

Presentation materials