WLCG DOMA BDT Meeting

Wednesday 6 Apr 2022, 16:30 → 17:30 Europe/Zurich

Brian Paul Bockelman (University of Wisconsin Madison (US)), Maria Arsuaga Rios (CERN), Petr Vokac (Czech Technical University in Prague (CZ))

Topic: WLCG DOMA BDT Meeting

New Zoom Meeting Link: https://cern.zoom.us/j/69074333781?pwd=aVRSbFMxZnF1MkdIeGF3MWdXM0VNUT09
New Meeting ID: 690 7433 3781
New Passcode: 95243662

16:30 → 16:35 Hot topics

16:35 → 16:50 Token Authorization testbed

Speaker: Francesco Giacomini (INFN CNAF)

Nightly tests:

- Nebraska, RAL are currently broken -- testbed nodes repurposed for other items.

- Bonn is looking quite good - one issue (storage.create still allows overwrites -- shouldn't do that!), definitely an XRootD bug.  They filed a ticket against xrootd.  Brian's not sure if this is an easy fix; Petr says it's an important use case for stageout.  Brian will prioritize.

- Rucio updates?  Waiting for a developer to start.  Not much progress.

- FTS: Nothing to report.  Need Rucio pieces first.

- Petr: What about xrootd + tokens?

   - Brian, Al: First-party works.  Server-side only change (slightly older clients are OK); not released by either dCache or XrootD.  xrootd-TPC approach with tokens is not agreed upon.  Al got one approach working but it's not clear whether that's OK with all involved.  Brian has not tested this with gfal2; if lucky, we shouldn't need gfal changes as this is all internal to the xrootd client.

16:50 → 17:05 Tape REST access

Speaker: Cedric Caffy (CERN)

• gfal progress with TAPE HTTP support tracked in DMC-1301
• REST API specifications under review
• Mihai notes that the exact status listed in the ticket might be slightly behind from reality.  Certainly in-progress.
• Steven notes that they are getting FTS devs test endpoints.
• Petr: What's the timescale?
  • Mihai: Plans to have it by Run3.  Hopefully there for CTA
  • Al: Coming up -- will be a layer on top of the existing dCache bulk API.  Bulk API is being refactored (patches being reviewed).  The wrapper layer -- couple of weeks?
• Steven Murray: Is anyone waiting for this urgently?
  • B: not clear.  ATLAS & CMS can survive for awhile on SRM+HTTP (at least through end of year perhaps?).  LHCb plans are unknown.
  • Paul: REST API is a prerequisite for non-X509 transition in some implementations (SRM may be heavily tied to X.509; not all).
  • Petr: What about OSG 3.6 / Globus Toolkit?  Brian: Again, a reason to do that this year -- but not clear this makes it urgent.  Most of the FTS servers that would be affected are not using Globus from OSG.

17:05 → 17:20 Packet marking

Speakers: Marian Babik (CERN), Shawn Mc Kee (University of Michigan (US))

- WG meeting took place on 24th of March (https://indico.cern.ch/event/1141264/)
   - Discussed collector design and architecture and proposed two possible ways forward

- WG udpate was presented at the LHCOPN/LHCONE workshop (https://indico.cern.ch/event/1110783/)

- Contacted all WLCG computing coordinators requesting a list of activities to be tracked (in progress; new activities were added to the registry based on responses we received so far; registry is now fully functional)

- Flow and Packet Marking Technical Specification (attached): Contains proposals on protocol extensions that would make it possible to propagate experiment ids and activities from DDMs to storages (open to comments and suggestions). 

- XRootD not sending flow markings to the original destination (only to a dedicated endpoint/collector): was followed up with Andy and should be fixed

- dCache flow marking meeting to be scheduled

- Next WG meeting will be beg. of May and will focus on packet marking (RFC review, eBPF/TC implementation, HW accounting)

• Petr: What's the overview of the UDP fireflies (flow marking)?
  • Marian: All agreed upon.  Doing the end-to-end coordination.  Integration activity at this point.
  • Marian: Packet marking requires more work.  Not all implementations (dCache) have direct access to the relevant kernel facilities.  IPv6-only.  RFC is changing.
  • Marian: For flow marking, ready to meet with the dCache team again... lots of progress over the last months.
• Petr: What's the status of the UDP firefly collectors?  Is it one at JANET, one at ESNet?  Do we need more?
  • Marian: load on the ESNet collector is good shape.  Making a proposal with the R&Es on how to get more collectors along the routes.  Will need hardware in-line.  Lots of discussion in progress -- too early to report on details.
    • Software container is quite a simple thing -- packets are syslog format so this is just syslog + file beats in a docker container.

17:20 → 17:30 AOB