19–25 Oct 2024
Europe/Zurich timezone

Data Movement Manager (DMM) for the SENSE-Rucio Interoperation Prototype

22 Oct 2024, 17:09
18m
Room 1.B (Medium Hall B)

Room 1.B (Medium Hall B)

Talk Track 1 - Data and Metadata Organization, Management and Access Parallel (Track 1)

Speaker

Aashay Arora (Univ. of California San Diego (US))

Description

The data movement manager (DMM) is a prototype interface between the CERN developed data management software Rucio and the software defined networking (SDN) service SENSE by ESNet. It allows for SDN enabled high energy physics data flows using the existing worldwide LHC computing grid infrastructure. In addition to the key feature of DMM, namely transfer-priority based bandwidth allocation for optimal network usage; it also allows for the identification of the exact cause of underperforming flows using end-to-end monitoring of the data flows by having access to host (network interface) level throughput metrics and transfer-tool (FTS) data transfer job level metrics. This paper describes the design and implementation of DMM.

Primary authors

Aashay Arora (Univ. of California San Diego (US)) Diego Davila Foyo (Univ. of California San Diego (US)) Justas Balcas (California Institute of Technology (US)) Thomas Lehman (ESnet) Xi Yang (LBNL)

Presentation materials