Nov 4 – 8, 2019
Adelaide Convention Centre
Australia/Adelaide timezone

Named Data Networking based File Access for XRootD

Nov 7, 2019, 3:30 PM
1h
Hall F (Adelaide Convention Centre)

Hall F

Adelaide Convention Centre

Poster Track 4 – Data Organisation, Management and Access Posters

Speaker

Jean-Roch Vlimant (California Institute of Technology (US))

Description

We present an NDN-based XRootD plugin and associated methods which have been built for data access in the CMS and other experiments at the LHC, its status and plans for ongoing development.

Named Data Networking (NDN) is a leading Future Internet Architecture where data in the network is accessed directly by its name rather than the location of the host where it resides. NDN enables the joint design of multipath forwarding and caching to achieve superior latency and failover performance. The Caltech team together with Northeastern University and other collaborators in the SDN Assisted NDN for Data Intensive Experiments (SANDIE) project has implemented a corresponding NDN naming scheme in order to access datasets, where the naming structure follows the format /ndn/xrootd/<file system call>/<filename>/<segment no><additional info>, and an Open Storage System (OSS) plugin for XRootD.

The XRootD plugin is a consumer entity that translates each filesystem call into an Interest Packet and sends it over the network using a local or remote NDN forwarder to find the corresponding data file from caches or producers. Alongside the plugin, a corresponding producer has been implemented as a service running on CentOS systems. The producer is capable of communicating with multiple file systems (HDFS, CEPH) and on receiving Interests it responds with data encapsulating a byte range at an offset from an existing file, specified in the name of the Interest packet, or an error code specifying the failure that occurred while trying to access it, e.g. the non-existence of a file containing the byte range requested.

In this paper we present the architecture of the NDN-based OSS plugin for XRootD and its implementation details. In addition, first results of transfer tests and data access comparisons are presented along with details about the setup, and future plans for software, file management and platform development.

Consider for promotion Yes

Primary authors

Justas Balcas (California Institute of Technology (US)) Catalin Iordache (California Institute of Technology (US)) Ran Liu (Northeastern University (US)) Jean-Roch Vlimant (California Institute of Technology (US))

Co-authors

Harvey Newman (California Institute of Technology (US)) Prof. Edmund Yeh (Northeastern University (US)) Raimondas Sirvinskas (California Institute of Technology (US))

Presentation materials

There are no materials yet.