Focus on:  Hide Contributions
Back to Conference View
Login

XRootD and FTS Workshop @ STFC UK

→ Europe/London
The Cosener's House

The Cosener's House

Abingdon UK
Alastair Dewhurst (Science and Technology Facilities Council STFC (GB)), Andrew Bohdan Hanushevsky (SLAC National Accelerator Laboratory (US)), Guilherme Amadio (CERN), Katy Ellis (Science and Technology Facilities Council STFC (GB)), Luca Mascetti (CERN), Mihai Patrascoiu (CERN)
Description

The XRootD and FTS workshop brings together the XRootD and FTS developers and people from Academia, Research, and Industry to discuss current and future data access activities as related to the XRootD framework and to the FTS project.ย 

Presentations focus on achievements, shortcomings, requirements and future plans for both XRootD and FTS projects.

Registration will open on 9th May (4 months before the event) and close on 23rd August (2 weeks before the event) . The cost for the workshop is ยฃ200 for the five days, it will include lunches and coffee breaks as well as a welcome reception. There will be the possibility to join the dinner reception for ยฃ70.

The workshop will take place at Cosener's House, Abingdon UK.

IMG_3067.jpg
Picture 1.jpg
Screenshot 2024-09-13 at 11.29.42.png
Videoconference
XRootD and FTS Workshop @ STFC UK
Please log in and register
Zoom Meeting ID
67861414414
Host
Mihai Patrascoiu
Useful links
Join via phone
Zoom URL
Registration
Registration Form
Apply for participation
  • Monday 9 September
  • Tuesday 10 September
    • FTS Presentations

      Overview Talks for FTS

      • 7
        FTS@CERN: Site Report

        General overview of the FTS deployment at CERN

        Speaker: Steven Murray (CERN)
        cern_fts_site_report_xrootd_and_fts_workshop_2024.pdf
      • 8
        Site update on FTS at RAL

        General overview of the current status of the FTS service being run at RAL and plans for it moving forward.

        Speaker: Rose Cooper
        fts site update ral.pdf
        fts site update ral.pdf
        fts site update ral.pptx
      • 9
        BNL FTS Status

        Report of BNL FTS status

        Speaker: Hironori Ito (Brookhaven National Laboratory (US))
        BNL FTS Status 2024.pdf
      • 10
        The Square Kilometre Array: SRCNet v0.1 and synergies with HEP

        The Square Kilometre Array (SKA) Observatory will be supported by a global network of SKA Regional Centres (SRCNet) distributed across its member states. SRCNet v0.1 โ€“ to be deployed in 2025 - represents the prototype compute, storage and service infrastructure needed to prepare for full operations.
        For SRCNet v0.1, Rucio, FTS and Storage endpoint technologies from the HEP Community have been selected to full the data management and logistics requirements.
        This presentation will provide an overview of the current status of SRCNet and outline the development and implementation plans for SRCNet v0.1,(e.g. data movement campaigns). Additionally, the talk will explore synergies with the HEP community and discuss potential future requirements as SRCNet develops.

        Speaker: James William Walder (Science and Technology Facilities Council STFC (GB))
        2024Sep09_FTSXRootD_SRCNet_final.pdf
    • Coffee break
    • FTS Presentations

      Overview Talks for FTS

    • FTS & DMC discussion (Future of FTS, DMC direction)
    • Lunch
    • Experiment input to FTS
    • Coffee break
    • FTS & XRootd: Meet the Developers
  • Wednesday 11 September
    • FTS Presentations

      Overview Talks for FTS

      • 12
        FTS in the Token World

        Overview of FTS in the token ecosystem, reflections on the DC'24 and decisions moving forward

        Speaker: Mihai Patrascoiu (CERN)
        FTS_XRootD_Workshop_2024_FTS_In_The_Token_World.pdf
      • 13
        FTS & Rucio

        The Rucio communities depend on FTS to orchestrate site-to-site transfers.

        Over the past year, the two development teams have worked closely to drive the transition from X.509 certificates to OAuth 2.0 tokens. This talk will focus on that effort. It will cover the original design, the preparation leading up to the Data Challenge 2024, the Data Challenge itself and the lessons learned from it, the more recent discussions and experiments, and a view into the short- and medium-term future.

        This talk will also mention other non-token-related developments in Rucio.

        Speaker: Dimitrios Christidis (CERN)
        2024.09.11 FTS & Rucio.pdf
    • Coffee break
    • FTS Presentations

      Overview Talks for FTS

      • 14
        CMS Data Transfers: An Overview of Token Workflows

        Since Run 1, CMS has relied on certificates for user identification and experiment/group membership through extensions. However, as support for both certificates and extensions declines, CMS is transitioning to token-based authentication, aligned with the WLCG profile, for the upcoming High-Luminosity LHC run. With certificates, sites were responsible for mapping roles to capabilities. Tokens will allow CMS to take a more granular approach to security whilst also aligning with industry-standard practices.
        This presentation provides an overview of the token-based authentication and authorisation workflows implemented for CMS data transfers. It also reflects on past experiences, outlines the current testing efforts, and discusses future enhancements to optimize the use of tokens within CMS.

        Speaker: Rahul Chauhan (CERN)
        cms_token_workflows.pdf
    • FTS & XRootd: Token Discussion
    • Welcome & Logistics
    • Lunch
    • Working Lunch: XRootD Stakeholder discussion and communications
      Convener: Andrew Bohdan Hanushevsky (SLAC National Accelerator Laboratory (US))
      XrootDworkshop24talk.pdf
      XWS-XRootD Project.pdf
    • XRootD Presentations
      • 16
        Catching Up With XRootD

        Features and changes since the last workshop (5.5.4 to 5.7.1).

        Speaker: Andrew Bohdan Hanushevsky (SLAC National Accelerator Laboratory (US))
        XWS-2024-Features.pdf
      • 17
        Catching Up With XRootD (Part 2)

        In this contribution, we discuss infrastructure updates to XRootD and other development topics.

        Speaker: Guilherme Amadio (CERN)
        catching-up-with-xrootd-part2.pdf
      • 18
        XRootD Future Feature Plans

        A roundup of planned features and improvmments.

        Speaker: Andrew Bohdan Hanushevsky (SLAC National Accelerator Laboratory (US))
        XWS-2024-Futures.pdf
      • 19
        Contributing to XRootD
        • XRootD Development Model
        • Documentation and Submitting Patches
        • GitHub Actions Continuous Integration
        • Building and Running Tests Locally
        Speaker: Guilherme Amadio (CERN)
        contributing-to-xrootd.pdf
    • Coffee break
    • Workshop Reception Cherwell Boathouse

      Cherwell Boathouse
  • Thursday 12 September
    • XRootD Presentations
      • 20
        Pelican and the OSDF Overview

        Pelican and the OSDF Overview

        Speaker: Brian Paul Bockelman (University of Wisconsin Madison (US))
        Pelican-Overview-Xrootd-Workshop-2024.pdf
      • 21
        XCache news & upcoming changes [25 min]

        New resource monitoring infrastructure.
        Purge plugin support.
        Planned development:
        - extensions of resource monitoring, planned and possible
        - improvement of prefetching

        Speaker: Matevz Tadel (Univ. of California San Diego (US))
        XCache-TheNew-ResMon-n-Purging-Abingdon-2024.pdf
      • 22
        The Pelican XrdCl plugin

        The Pelican XrdCl plugin

        Speaker: Brian Paul Bockelman (University of Wisconsin Madison (US))
        Pelican-XrdCl-Xrootd-Workshop-2024.pdf
    • Coffee break
    • XRootD Presentations
      • 23
        Stress testing an OSDF/Pelican/XrootD origin

        We create a set of tests to test every hardware and software limit related to XrootD. The idea of this talk is to show the results of these tests. Moreover, it will show some OSDF statistics.

        Speaker: Fabio Andrijauskas (Univ. of California San Diego (US))
        XrootD24.pdf
      • 24
        The Pelican Globus/HTTP/S3 OSS backend

        The Pelican Globus/HTTP/S3 OSS backend

        Speaker: Brian Paul Bockelman (University of Wisconsin Madison (US))
        Pelican-Backends-Xrootd-Workshop-2024.pdf
      • 25
        Experince with XCache on HPC in Germany (CMS)

        With this contribution, I want to present our first production deployment of XCache for workflow and efficiency optimizations of CMS jobs at our local HPC cluster at KIT, HoreKa. The project is part of the preparations for the future German HEP computing strategy focusing on HPC contributions.
        Our fully containerized setup is deployed on a login node of the cluster and uses a shared filesystem for the cache (GPFS with RDMA/IPoIB) to optimize the data access for the cluster.
        The talk will include a description of our setup, the challenges and limitations with XRootD on HPC we encountered, and the experiences we made.

        Speaker: Robin Hofsaess (KIT - Karlsruhe Institute of Technology (DE))
        RHofsaess_XRD24-2.pdf
      • 26
        OSCER Status
        Speaker: Horst Severini (University of Oklahoma (US))
        OU-xrootd-2024.pdf
    • Lunch
    • XRootD Presentations
      • 27
        UK storage with XRootD

        UK Storage (XrdCeph, CephFS+XrooTD, XCAche, and VP)

        Speaker: Alastair Dewhurst (Science and Technology Facilities Council STFC (GB))
        XRootDWorkshop20240912.pdf
        XRootDWorkshop20240912.pptx
      • 28
        CERN Central Monitoring Overview

        A presentation on CERN Central Monitoring, showcasing how data gets ingested, processed, enriched, aggregated and stored in OpenSearch.

        From OpenSearch storage, tools such as Grafana can leverage the data and create dashboards and plots.

        Speaker: Borja Garrido Bear (CERN)
        2024-09_12_CERN_Monitoring_overview.pdf
      • 29
        Shoveler XRootD monitoring

        The old XRootD monitoring 'GLED' has been turned off. It will be replaced with Shoveler. This presentation looks at the testing and validation of this software, as well as other XRootD monitoring status.

        Speaker: Katy Ellis (Science and Technology Facilities Council STFC (GB))
        Shoveler XRootD workshop 2024.pdf
    • Coffee break
    • XRootD Presentations
      • 30
        XRootD monitoring at Lancaster

        We show how we've combined three means of monitoring of our gateways, and suggest some enhancements.

        Speaker: Steven Simpson (Lancaster University)
        lancs-XRootD-monitoring-2024.pdf
      • 31
        XRootD Monitoring Streams

        The types of information avalable via XRootD Monitoring

        Speaker: Andrew Bohdan Hanushevsky (SLAC National Accelerator Laboratory (US))
        XWS-2024-Monitoring.pdf
      • 32
        Discussion: XRootD Monitoring: What's missing? What needs to change?

        Discussion

        Speaker: Andrew Bohdan Hanushevsky (SLAC National Accelerator Laboratory (US))
  • Friday 13 September
    • XRootD Presentations
      • 33
        Network isolation for multi-IP exposure in XRootD

        Outline:
        - The need to expose multiple IPs
        - Network namespaces for isolation
        - Manual approach
        - k8s/multus approach

        Duration: 20 min Virtual Presentation

        Speaker: Diego Davila (UCSD)
        Network Isolation for multi-IP exposure in XRootD.pdf
      • 34
        Optimising Vector Read requests for RAL disk storage

        For a long time Ceph-based disk storage at RAL Tier-1 was not able to execute Vector Read requests effectively, causing problems for some VOs. The talk describes multiple changes that were made to the Xrootd-Ceph plugin and configuration to solve the problem.

        Speaker: Alexander Rogovskiy (Rutherford Appleton Laboratory)
        vread_xrootd_workshop.pdf
      • 35
        XKIT for GridPP: (XRootD Kubentes Integration Testing for GridPP)

        XRootD is a robust, scalable service that supports globally distributed data management for diverse scientific communities. Within GridPP in the UK, XRootD is used by the Astronomy, High-Energy Physics (HEP) and other communities to access >100PB of storage. The optimal configuration for XRootD varies significantly across different sites due to unique technological frameworks and site-specific factors.

        XRootD's adaptability has made it a cornerstone of the national data-management strategy for GridPP. Given its high-profile role, new releases, and features of XRootD undergo rigorous testing and verification before national deployment. Historically, this process involved manual integration testing and dedicated test deployments, which required substantial input from both local site administrators and remote support teams. This approach has placed considerable demands on support staff, requiring extensive technical expertise and significant time for verification.

        To support the storage community within GridPP, we have developed a system that automates the deployment of a virtual grid using Kubernetes for XRootD testing, "XKIT". Using a container-based approach this system enables high-level integration tests to be performed automatically and reproducibly. This not only simplifies the support process but also significantly reduces the time staff need to dedicate to repetitive testing for new deployments.

        We have identified >20 unique XRootD configurations necessary for XKIT. By deploying each of these setups on our platform, we aim to provide the GridPP community with a consistent suite of functional tests tailored to various site topologies.

        This presentation will explore the development of the XKIT platform, discuss the challenges we encountered, and highlight the advantages this system offers to GridPP and the wider community.

        Speaker: Dr Robert Andrew Currie (The University of Edinburgh (GB))
        XKit3.pdf
      • 36
        Benchmarking XRootD: A Versatile Testing Framework for the Grid Community

        As Tier 1 storage continues to expand, an increasing number of sites are contributing to the Worldwide LHC Grid, making efficient data transfer a critical component for big data analytics. XRootD is pivotal for scientific data management, facilitating seamless data movement and access across the 3 tiers. However, with the growing complexity and scale of grid infrastructures, it is essential to benchmark and evaluate the performance of XRootD in its intended production environment.

        This talk introduces the XRootD testing framework, a tool designed to assess the performance of XRootD functionality across multiple tiers, networks and protocols. By enabling systematic benchmarking, this framework provides valuable insights into the functionality and efficiency of data transfers, ensuring that sites can test different configurations and their impact. The framework is highly extensible to other sites and functionalities, making it a versatile tool for the wider grid community. Test results can be run and visualised in Jenkins, which offers insights for developers. Attendees will gain an overview of the framework, and how to utilise it to measure performance and reliability of data transfers within their own grid institutions.

        Speaker: Mariam Demir
        XRootD Workshop - Benchmarking Test Framework v2 (3).pdf
        XRootD Workshop - Benchmarking Test Framework v2 (3).pptx
    • Coffee break
    • XRootD Presentations
      • 37
        Load Balancing in XRootD

        To address the need for high transfer throughput seen for large datacentres using XRootD for projects such as the LHC experiments, it is important to make optimal and sustainable use of our available capacity. Load balancing algorithms play a crucial role in distributing incoming network traffic across multiple servers, ensuring optimal resource utilization, preventing server overload, and enhancing performance and reliability. At the Rutherford Appleton Laboratory (RAL), the UK's Tier-1 centre for the Worldwide LHC Computing Grid (WLCG), we started with a DNS round robin then moved to XRootD's cluster management service component, which has an active load balancing algorithm to distribute traffic across 26 servers, but encountered its limitations when the system as a whole is under heavy load. We describe our tuning of the configuration of the existing algorithm before proposing a new tuneable, dynamic load-balancer based on a weighted random selection algorithm, as well as the observed behaviours of servers under stress conditions

        Speaker: Jyothish Thomas (STFC)
        XRootD Load balancing.pdf
      • 38
        XRootd Http as web server for ROOT [10 + 10 min]

        Present motivation & possibilities.
        Discussion on feasibility and required changes for prototype implementation.

        Speaker: Matevz Tadel (Univ. of California San Diego (US))
        XRootd-as-httpd-for-ROOT-Abingdon-2024.pdf
      • 39
        On the use of XRootD in StorageD at the Rutherford Appleton Laboratory, UK

        StorageD is the data aggregator component within archiving systems that supports the work of the Diamond
        Light Source (DLS) and the Centre for Environmental Data Analysis (CEDA) at the Rutherford Appleton
        Laboratory (RAL). StorageD provides file ingest and recall services to scientists and engineers internationally
        through DLS and CEDA. StorageD currently support ingest of over 100TB daily. StorageD uses the CERN
        Tape Archive (CTA) backend in writing data to tape. Current holding on tape is approximately 100PB.
        Over the past 2 years, StorageD has migrated from using the RFIO access protocol to using XRootD for file
        transfer to the backend. This talk will cover the description of the development work on the transition of
        StorageD away from RFIO at the backend, towards XRootD. Furthermore, optimisation work that has become
        necessary because of this transition will be discussed.
        Service management of StorageD includes monitoring, reporting and management with maintenance scripts.
        These efforts will be discussed in this talk.

        Speaker: Emmanuel Bejide
        On the use of XRootD in StorageD at the Rutherford Appleton Laboratory.pdf
        On the use of XRootD in StorageD at the Rutherford Appleton Laboratory.pptx
    • 12:30
      Lunch
    • XRootD Presentations
      • 40
        Deep Dive into XRootD Github Repo

        How to maneuver around the XRootD Github repository and find helpful treasures at every click.

        Speaker: Guilherme Amadio (CERN)
    • Workshop wrap-up
Powered by Indico v3.3.5-pre