WLCG DOMA BDT Meeting

Wednesday 22 Nov 2023, 16:30 → 17:30 Europe/Zurich

Brian Paul Bockelman (University of Wisconsin Madison (US)), Maria Arsuaga Rios (CERN), Petr Vokac (Czech Technical University in Prague (CZ))

Topic: WLCG DOMA BDT Meeting (twiki)

16:30 → 16:35 News

16:35 → 16:45 Tape REST access

Speaker: Mihai PATRASCOIU (CERN)

16:45 → 17:00 Transfers with tokens

Speakers: Petr Vokac (Czech Technical University in Prague (CZ)), Francesco Giacomini (INFN CNAF)

CERN FTS pilot & tokens - progress

• Now configured also to support ATLAS token issuer
  • successful transfers test between dCache (PRAGUELCG2) <-> EOS (CERN) <-> StoRM (INFN-T1)
  • audience was not configured correctly on EOS ATLAS (already discussed with EOS Ops)
  • different StoRM behavior for "stat" operation
• StoRM: storage.modify doesn't allow stat operations while this is allowed in dCache & XRootD
  • preferred solution
    • storage.create & storage.modify should allow "stat" operation
    • StoRM code needs to be updated
  • currently FTS needs storage.read + storage.modify when StoRM is HTTP-TPC destination
    • releasing fixed StoRM & deploying at sites takes time and DC24 is already quite close
    • Rucio will be able to use workaround with read+modify for DC24 transfers
  • consistency of WLCG JWT profile implementation is certainly the important thing
    • posix filesystems also would typically not require r for w
    • requiring read for "stat" is a bit confusing

Operational model for tokens during DC24

• CERN would like to know rates, who will be included, what are our plans with token related mini-challenges
  • asked for a document that can be shared between iterested parties
  • would like to understand both ATLAS and CMS plans
• Rucio developers mentioned situation will be clear within two weeks once we have initial implementation for full chain with tokens
  • ATLAS and CMS behavior should be same by design (not much flexibility in Rucio token configuration in initial release that will be used in DC24)

17:00 → 17:10 Packet marking

Speakers: Marian Babik (CERN), Shawn Mc Kee (University of Michigan (US))

Screen Shot 2023-11-16 at 2.56.08 PM.png

Screen Shot 2023-11-16 at 9.45.35 AM.png

Succesfull SC23 demo

- Packet marking @300Gbps (see screenshots attached) 
- Tested integration with ESnet high-touch service, dataset to be published later this week  
- InMon: SFlow-RT readout of the packet marking from network equipment worked fine (screenshot attached)

- New fireflies dashboard showing data from a production storage (at UNL): 
.   - https://public.stardust.es.net/d/b8dddac0-5b24-4739-9c8d-e88a05c1344f/scientific-network-tags3a-rande-dashboard?orgId=1&from=now-12h&to=now 
   - There is additional data available that is not yet shown both from fireflies as well as from Esnet, e.g. duration of the flows, transfer rates, etc.

Developments

- flowd - developed support for packet marking on multiple NICs (tested succesfully @ SC23) 
   - plan to create separate packages for flow labeling and packet marking (flowd and flowd-ebpf) - different deps + EPEL requirements 
   - following SC23 tests we will also refactor the way fireflies are processed and how additional data is added 
- xrootd http-tpc support was released 
- FTS, Rucio and gfal2  - waiting for deployment

dCache fireflies

• dCache AGLT2 tests with fireflies - discovered problems also in latest 9.2.5
• we hope that this could be fixed in next release (december?)
• enabling fireflies is simple - just flip one boolean value

XRootD & support for flowd activity

• available since XRootD 5.6.3 
  • already available in EPEL 
  • work in progress with OSG packages 
• this functionality is already included in EOS that is now being deployed at CERN 
• Rucio needs to be updated to pass activity for SciTags to FTS-1829
  • actually seems to be already implemented

17:10 → 17:25 WebDAV Error Message Improvement Project & unified error message format

Discuss with experts improvements in the error messages produced by failed transfers.
https://twiki.cern.ch/twiki/bin/view/LCG/WebdavErrorImprovement

Speaker: Stephan Lammel (Fermi National Accelerator Lab. (US))

Gfal upload / download disknode hostname (DMC-1390, rucio#5980), ATLAS would like to have these details in Rucio traces / Monit data. For HTTP-TPC we might get this info from performance markers in the future, but we would like to have this information available also for simple upload/download.

• development that's not directly related to DC24 will happen only after this event.
• it seems it should not be difficult to provide this functionality (but first FTS developers needs to think what would be the best way to provide real souce/destination hosts)

17:25 → 17:30 AOB