dCache with xRootD monitoring flow discussion

Friday 17 Feb 2023, 16:00 → 17:00 Europe/Zurich

Attended:

Borja, Albert, Dmitry, Christian, Tigran, Julia

Motivation:

Create global WLCG dashboard for data transfers and remote data access. Should cover all kind of traffic on the WLCG infrastructure, namely those which are handled by FTS and those which are initiated by remote reading via xrootd protocol. The latter case consists of native xroootd monitoring when monitoring data is reported by xrootd server and dCache with xrootd protocol on top. The latter has been the topic of discussion.

 

Time line:

Before WLCG DC in 2024

Scope:

WLCG sites which provide remote access via xrootd protocol to data hosted by dCache

Basic workflow:

• Every site which provides remote access via xrootd protocol to data hosted by dCache should enable reporting of billing info to Kafka and correspindigly setup a Kafka service
• There should be a special component which would translate dCache billing record into the one as described in the document: https://docs.google.com/document/d/1tBECfGHk4AybPoorpEe2WiBwYH9zodv-4shiW1RGUv4/edit and report the generated message to ActiveMQ at CERN. This component would not be a part of dCache release ( may be we can re-discuss it later?)

Actions:

• Cristian kindly agreed to create a document with mapping of attributes in dCache billing record into the generic monitoring format as mentioned above
• Develop a prototype to test the concept (reading form Kafka- translation-reporting to ActiveMQ). We did not decide today, who will work on prototype. Did we? But looks like a straight forward task for dcache experts.
• Borja will get in touch with ActiveMQ support at CERN to follow up on authentication and dedicated queue/topic creation and will provide info of how to report to message queue at CERN
• Julia will create a mailing list for people involved in this activity
• We will meet in couple of weeks or so to see where we are.

16:00 → 16:20 Introduction. What we need to do.

Speaker: Borja Garrido Bear (CERN)

common-monitoring-transfers.pdf

common-monitoring-transfers.pptx

16:20 → 16:40 Discussion