27 August 2023 to 10 September 2023
University of Sao Paulo, USP, Brazil
America/Sao_Paulo timezone

Extreme scale dataflows in the compute continuum for the next generation of giant astronomical observatories

4 Sept 2023, 11:20
50m
University of Sao Paulo, USP, Brazil

University of Sao Paulo, USP, Brazil

Speaker

Prof. Damien Gratadour (Observatoire de Paris (FR))

Description

Abstract: The next generation of multi-science hubs such as the SKA are based on a highly challenging operating principle: they generate extreme scale data volumes to be processed and reduced in a guaranteed maximum time, while remaining as energy efficient as possible and operable for decades. To build these infrastructures, the complete digital continuum has to be realized, through a hierarchy of cyber-infrastructures, fed continuously by science sensors at the edge ; filtering, combining, processing and reducing continuous data streams in quasi real-time locally on supercomputers; and generating science grade data products to be delivered worldwide to a distributed community of scientists, relying on cloud infrastructures for high-level analysis. The challenges faced by international scientific communities in looking towards the implementation and exploitation of such experiments are driven by i) a paradigm shift to a dataflow model in which data streams have to be reduced in real-time before being discarded, ii) managing a new scale of data volumes, up to the multi-Exabyte as well as iii) fitting development, operations, maintenance and upgrades within a restricted cost and power envelope. In this talk, I will cover more quantitatively these challenges and will present the large scale R&D initiatives initiated at national and European levels to address them, federated under joint public-private laboratory: ECLAT.

Lecturer: Senior Research Scientist at Observatoire de Paris, CNRS.
Damien holds a PhD in Observational Astronomy from Université Paris-Diderot (2005). He has been an Adaptive Optics (AO) fellow, responsible for the last stages of commissioning of the Altair AO system on the Gemini North Telescope in Hawaii (2006) ; and an Instrument Scientist (2007-2008), for GeMS, the Gemini MCAO System, a facility featuring 6 Laser guide stars.
Since 2008, at Observatoire de Paris - PSL, Damien has been leading an original research program on high performance numerical techniques for astronomy including modeling, signal processing and instrumentation for large telescopes. He has been the P.I. of several large programs at national and European levels targeting AO Real-Time Controllers for giant optical telescopes with emerging computing technologies. Since 2021, with France officially joining SKAO, he is also getting strongly involved in the French effort dedicated to the construction of this giant radio-telescope. In particular, he is currently coordinating the implementation of ECLAT, a joint laboratory between CNRS, INRIA and Atos, as a long-term support structure federating resources from academic and industrial teams that will engage in the co-design work for the French contribution to the SKA (Text informed by the Lecturer).

Presentation materials