Speaker
Description
DUNE is a next-generation neutrino oscillation experiment. During its decades-long operational lifetime, it is expected that many exabytes of data will be collected. It is critical that this data be correctly characterized with respect to its associated conditions metadata – the non-event data used to process event data during reconstruction and analysis. To meet the operational scale and reliability requirements for DUNE’s conditions metadata management, a new HEP metadata exchange platform – Metadex – is being built.
Using the HSF Conditions Database Reference Implementation as a guide, Metadex is designed to be highly available, performant, scalable, secure, maintainable, and portable. It will provide automated offline data workflow orchestration utilizing distributed and message-driven architectural patterns. Industry standard technologies will be used in its component stack: FastAPI (API application server), PostgreSQL (DBMS), CVMFS (payload file storage), Redis (caching and messaging), and Kubernetes (containerized workloads). While the initial focus is on supporting DUNE conditions metadata use cases, the intent for Metadex is to be HEP experiment agnostic and extensible to support a variety of metadata management uses cases.