Focus on:  Hide Contributions
Login

XRootD Collaboration Meeting

→ Europe/Zurich
31/3-004 - IT Amphitheatre (CERN)

31/3-004 - IT Amphitheatre

CERN

105
Show room on map
Andrew Bohdan Hanushevsky (SLAC National Accelerator Laboratory (US)), Guilherme Amadio (CERN)
XRootD Collaboration Meeting
Zoom Meeting ID
62381053456
Host
Guilherme Amadio
Useful links
Join via phone
Zoom URL
Hide

XRootD Collaboration Meeting Minutes

This was a collaboration meeting for the XRootD project team to discuss recent developments and future plans with stakeholders and the user community in general. The team reviewed XRootD 6 releases, which included security fixes, OpenSSL 4.x support, and the new XrdOssMirage plugin developed by Diogo that simulates file storage without actual disk usage. Guilherme presented a new monitoring evolution prototype with native support for Prometheus and OpenTelemetry metrics, which provides detailed performance tracking and JSON log streaming capabilities. Diogo discussed HTTP/TPC support implementation for XRootD clients, including copy job functionality, perf marker integration, and token authentication features, which will be released in XRootD 6.2. The team reviewed WLCG OTF#9 items and discussed documentation evolution challenges, noting that converting their extensive professional documentation would require significant manual effort. Guilherme presented AI usage policies and recommendations for the XRootD project, covering best practices for using coding assistants like Claude and GitHub Copilot while maintaining code quality and security. The conversation ended with a discussion about naming for a new command line tool that would combine xrdfs and xrdcp functionality to replace gfal2-util, with the team deciding to explore extending xrdfs rather than creating a new separate tool.

Summary

Participants

Andreas Peters, Andrew B. Hanushevsky, Anil Panta, Cedic Caffy, David Smith, Diogo Mattioli, Guilherme Amadio, Gustavo Mattos Lopes, Jyothish Thomas, Latchezar Betev, Leopoldo Muñoz, Lorena Lobato, Luca Mascetti, Luis Obis Aparicio, Martina Samraj Solomon, Rahul Chauhan, Sam Skipsey, Stephan Lammel, Yujun Wu.

XRootD Security and Updates

Guilherme presented updates on XRootD releases, highlighting recent security vulnerability fixes and improvements to the continuous integration system, and the fact that XRootD is following as much as possible the Open Source Security Foundation best development practices. XRootD 6.0 moved to C++20, but only what is supported in GCC 8.5, due to ABI stability concerns which would come from using a more recent devtoolset compiler on Alma 8, which needs to remain supported. XRootD 6.0 also dropped support for old OpenSSL versions (1.0.x), and added ABI checks to ensure backward compatibility. XRootD 6.1 introduced support for OpenSSL 4.x, a new XrdOssMirage plugin, and initial streaming real-time checksum support, with many bug fixes also implemented.

XRootD's new XrdOssMirage Plugin in XRootD 6.1

Diogo Mattioli demonstrated XrdOssMirage, a server-side plugin that simulates files without actual storage, which is already included in XRootD 6.1 and being used for testing and benchmarking.

Network Simulation and HTTP TPC

Diogo clarified that the network simulation feature will copy bytes over the network but doesn't store them on disk, allowing for testing of network bottlenecks without actual data storage. The team discussed the implementation of HTTP TPC support for clients, including new command line arguments for token management and push/pull selection, with the copy job functionality already delivered in XRootD 6.2.

Review of WLCG OTF#9 Action Items

The meeting included a review of WLCG items, with most related to data transfers marked as complete or in progress, though some items like token challenges and documentation evolution remain unclear and require further clarification from WLCG OTF on what is expected.

Documentation and Testing Initiatives

Andy discussed challenges with documentation accessibility and editing, proposing to use CERNBox with Microsoft Word for collaborative editing, though this would require significant manual effort.

Test Pyramid and AI Policies

Diogo presented on test pyramid implementation, explaining the hierarchy from unit tests (fast, granular) to integration tests to end-to-end tests (slower, broader), and outlined best practices including single assertions per test, no logic in tests, meaningful names, static inputs, and test independence. The team then shifted to discussing a testing initiative within the XRootD project, aiming to eliminate boilerplate code, prevent regressions, make failures easy to diagnose, and increase team engagement through available testing frameworks like gtest and gmock, and for end-to-end tests, the BATS framework.

XRootD AI Policies and Recommendations

The team discussed AI usage policies, with Guilherme proposing guidelines requiring developers  to understand AI-generated code, include disclosure in commits, and verify changes independently. The intent is to build trust between developers, whether using AI or not, and prevent slowing down the development process or introducing hard to debug regressions due to the usage of AI.

Next steps

Diogo

  • Implement the macaroons transparent generation feature for HTTP TPC in xrdcp.
  • Implement the "--header" command line argument for passing additional headers in xrdcp.
  • Continue working on the testing initiative, including finalizing the gmock and BATS framework integration.

Guilherme

  • Finalize and release the official Docker image for XRootD before the workshop, targeting XRootD 6.2.
  • Refine and prepare the new monitoring prototype (Prometheus/OpenTelemetry) for production deployment.
  • Work with WLCG OTF to clarify the requirements for the "token challenge" and other unclear items.
  • Formalize and document the AI usage policy for XRootD contributions.

Luis

  • Finalize the implementation of the HTTP tape REST API, ensuring it uses the plugin's thread pool for requests.
  • Finalize the implemenation of equivalent functionality of gfal2-util into xrdcp/xrdfs command line utilities
There are minutes attached to this event. Show them.