The [Vera C. Rubin observatory][1] is preparing the execution of the most ambitious astronomical survey ever attempted, the Legacy Survey of Space and Time. Due to start operations late 2024 for 10 years, it will nightly scan the southern sky and collect images of the entire visible sky every 4 nights. Detection of celestial objects will be performed on those images to progressively build an...
CTA is the CERN Tape Archive solution in production for LHC Run-3.
This presentation will focus on the current CTA production deployment and will provide an up-to-date snapshot of CTA achievements since the start of Run-3.
It will also cover production deployment plans.
The good old Rucio WebUI has served our community well over the last several years. However, it lacks the architecture, frameworks, and technologies to sustain it over the next decade. That's where the efforts to revamp the old WebUI step in. This talk discusses the plan, progress, and what the users can expect from the new Rucio WebUI.
Multi-Vo Rucio allows a single instance of Rucio serve several VOs. Each VO is separated into its own namespace, providing complete separation between the VOs data, and each VO with their own customisability with policy packages.
Work is underway to ensure Multi-VO capabilities for the new WebUI, and integration with DIRAC workflow management software.
The Multi-VO instance of Rucio is...
This document presents the work done during the Escape’s project by the CTA team in order to evaluate the integration of the ESCAPE’s Data Lake with the Workload Manager (DIRAC).
The work was distributed across differents teams:
-PIC in Barcelona was in charge of providing the Data Lake and storage infrastructure
-DIRAC’s team has set up the DIRAC instance in CC-IN2P3
-LAPP was in charge...
Metadata management is one of three major areas of scientific
data management along with replica management and workflow
management. Metadata is the information describing the data stored in a data
item such as a file or an object. It includes the data item provenance, recording
conditions, format and other attributes. MetaCat is a metadata management
database designed and developed for...
The DUNE Distributed Data management currently has 16 Rucio Storage Elements spread around the world, all currently using X.509 authentication. We have a local physics groups storage deployed at Fermilab that is using token authentication. We will present our current schema for file permissions at distributed storage elements, and how we expect this to change in the era of tokens. We will...
As distributed scientific collaborations reach the Exabyte scale, moving data becomes more challenging and thus, the capacity of managing priorities among the different transfer requests over the network becomes a necessity.
Using network services to build bandwidth guaranteed paths and fixed routes can be used to prioritize transfer requests.
We have built a system that creates such...
Analyzing the HEP traffic flows in detail is critical for understanding how the various complex systems developed by the LHC experiments are actually using the network. In this talk we will describe the work of the Research Networking Technical Working Group in the area of packet marking and flow labeling. The goal is to enable identification of any research and education network traffic...
The transfer subsystem of Rucio, also known as 'conveyor', was heavily reworked in the past year. This talk starts with a high level overview of Rucio transfers, followed by a deep dive into architectural changes and improvements implemented in recent Rucio releases.
FTS presentation about the current state of affairs and planned changes for 2023 and after.
In this talk we are going to describe the operational and conceptual design of the bulk archive management system involved in prototyping activities of the Cherenkov Telescope Array Observatory (CTAO) Bulk Archive. This particular archive in the CTA Observatory will take care of storage and management of the lower data level products coming from the Cherenkov telescopes, incuding their...
The Science Data Centre (SDC) is a new strategic infrastructure at the Leibniz-Institut für Sonnenphysik (KIS) in Freiburg, Germany, for archiving and disseminating raw and calibrated ground-based high-resolution multiwavelength spectropolarimetric and imaging data obtained primarily at the German Solar Telescopes in Tenerife, Spain, and soon from other observatories. Additionally, SDC...
