11–15 Mar 2024
Charles B. Wang Center, Stony Brook University
US/Eastern timezone

The Workflow Management System for Data Processing towards Photon Sources

11 Mar 2024, 16:15
30m
Charles B. Wang Center, Stony Brook University

Charles B. Wang Center, Stony Brook University

100 Circle Rd, Stony Brook, NY 11794
Poster Track 2: Data Analysis - Algorithms and Tools Poster session with coffee break

Speaker

lei wang (Institute of High Energy Physics)

Description

The new-generation light sources, such as the High Energy Photon Source (HEPS) under construction, are one of the advanced experimental platforms that facilitate breakthroughs in fundamental scientific research. These large scientific installations are characterized by numerous experimental beam lines (more than 90 at HEPS), rich research areas, and complex experimental analysis methods, leading to many data processing challenges: high-throughput multi-modal data, flexible and flexible and diverse scientific methodology, and highly differentiated experimental analytical processes. For a long time, there has been a lack of a general, user-friendly, and fully functional experimental data processing full-process management system in China and abroad. This project will use the idea of "workflow" to independently design and implement a set of graphical general-purpose management systems to solve the following key problems: how to quickly share and apply data processing methods to experiments by beam-line scientists, experiment users, and methodology developers during analysis; how researchers can flexibly customize and monitor complex and diverse data processing processes; how the whole process of experimental analysis can be applied in batches to similar experiments and the results can be reproduced. Two typical photon source experiments, the acquisition of the pair distribution function (PDF) of diffraction scatterings and the structural analysis of biological macro-molecules, will be used as application examples to show us how the workflow management system facilitates scientific research.

Significance

For a long time, there has been a lack of a universal, user-friendly, and comprehensive experimental data processing end-to-end management system, both domestically and internationally. Employing the workflow as the central tool offers an excellent solution by establishing connections between algorithm developers and experimental users. Moreover, it serves as a bridge linking hardware facilities and various modules within data processing software.

Primary authors

Dr Hao-Kai Sun (IHEP, CAS) Yu Hu

Co-authors

FU Shiyuan fusy Jianli Liu (Institute of High Energy Physics) Qi Fazhi (IHEP) Rui Liu (Institute of High Energy Physics) lei wang (Institute of High Energy Physics)

Presentation materials