17–24 Jul 2024
Prague
Europe/Prague timezone

Design of a Distributed Parallel Computing Engine for High-Throughput Data Processing of HEPS

18 Jul 2024, 19:00
2h
Foyer Floor 2

Foyer Floor 2

Poster 14. Computing, AI and Data Handling Poster Session 1

Speaker

Mr yu hu

Description

As a fourth-generation synchrotron light source, the experiments carried out by HPES will be shifted to high throughput, multi-modal, ultra-fast frequency, and cross-scale forms. The annual data flux produced by the experiment is expected to enter the "Exa-scale" era. Faced with such high-throughput experimental data, the arithmetic power of a single computing node is difficult to meet the needs of data analysis. This brings up the need to develop a high performance, robust and user friendly distributed parallel computing engine for data processing software.Due to the large difference in data rates between different beamline stations, how to support heterogeneous resources (GPUs) in a flexible and fine-grained manner becomes a challenge.To accelerate the computation and large datasets process, we design a distributed parallel computing engine capable of invoking scalable distributed heterogeneous computing resources to provide computational analysis services at different scales.

I read the instructions above Yes

Authors

Presentation materials