Help us make Indico better by taking this survey! Aidez-nous à améliorer Indico en répondant à ce sondage !

19–25 Oct 2024
Europe/Zurich timezone

Design and construction of High Energy Photon Source (HEPS) scientific data storage system

WED 14
23 Oct 2024, 15:18
57m
Exhibition Hall

Exhibition Hall

Poster Track 1 - Data and Metadata Organization, Management and Access Poster session

Speakers

Qingbao Hu (IHEP) Yaosong Cheng (Institute of High Energy Physics Chinese Academy of Sciences, IHEP)

Description

The High Energy Photon Source (HEPS) in China will become one of the world's fourth-generation synchrotron light sources with the lowest emittance and highest brightness. The 14 beamlines for the phase I of HEPS will produces about 300PB/year raw data, posing significant challenges in data storage, data access, and data exchange. In order to balance the cost-effectiveness of storage devices and realize the high reliability of data storage, a three-tier storage is designed for storing experimental data, including beamline storage, central storage, and tape. Raw data and processed data are stored on the beamline storage for a maximum of 7 days, on the central storage for a maximum of 90 days, and only the raw data are archived to tape for long-term storage with two copies. Of course, this data storage policy could be adjusted according to the actual data volume and funding situation of HEPS. The beamline storage utilizes a distributed all-flash SSD array to achieve high data input/output speeds. The central storage utilizes a distributed high-density HDD array to achieve medium to high-speed data IO. The tape storage complies with the LTO9 standard.
In addition, we have conducted some personalized optimizations based on the requirements of the HEPS project, such as adapting the Lustre file system to the Roce network protocol, and mapping permissions for users from AD domain control and LDAP domain control.

Primary author

Yaosong Cheng (Institute of High Energy Physics Chinese Academy of Sciences, IHEP)

Co-author

Presentation materials