WLCG DOMA Network Data Challenges

• Provisional date: last week of September (Rucio DB update could move it only by a week since last 2 weeks of October host test beams).

  • Window for the challenges is now first 2 weeks of October

• Central Infrastructure ready to submit transfer with label --activity “Data Challenge”.

  • Functional tests tried with all ATLAS SCRATCHDISK sites using:
    `rucio01:wlcg_doma_dc_test.0001` container, that includes only
    mc15_13TeV:EVNT.10267991._000001.pool.root.1
    Total files : 1
    Total size : 150.365 MB
    Total events : 5000

  • Results available at prototype of common monitoring (Data Challenges/FTS Status Board - https://monit-grafana.cern.ch/d/ZqU5ugjMz/fts-status-board?orgId=20 ).

• Two different approaches to consider:

  • A1: to test link siteA-siteB, files present at siteA but not at siteB should be identified and the related dataset attached to a new dedicated container (`rucio01:wlcg_doma_dc_test.0001`).

  • A2: existing files should be replicated into different `scope:name` pair copies and later deleted.

• Experiments feedback and status:

  • ATLAS: ongoing effort to efficiently integrate machinery for A1.

  • CMS: central infrastructure waiting for feedback (after dedicated meeting) to identify cms-responsible and preferred approach. Still under discussion if central infrastructure to be used or not. CMS needs to upgrade to at least Rucio-release 1.25.6 to profit from --activity “Data Challenge”. CMS needs to upgrade to at least Rucio-release 1.25.6 to profit from --activity “Data Challenge”. Proposal for A2 as something similar has already been done.

  • LHCb: agreed to integrate --activity “Data Challenge” label as well (at FTS level). Will NOT use central infrastructure.

  • ALICE: participating in the xrootd monitoring discussions.

Data Challenges general

• Common meeting with TAPE challenges and T0 challenges decided to cooperate to avoid stepping on each other with too many requests
  • We think that the two challenges should be technically kept separated i.e. the data challenges should not involve tape
    • Should also happen after the network challenges because they measure an extra layer
      • Alice only interested in tape challenges
      • LHCb not clear
  • Tape information requested to T1 sites https://twiki.cern.ch/twiki/bin/view/LCG/TapeTestsPreparation
• xrootd monitoring review
  • Meeting to review the xrootd monitoring
    • Replace current gled collector with new collector
    • Add a UDP -> TCP packet translator to avoid loosing information over WAN
    • There is almost an agreement but this might become a medium term activity
    • xrootd fields may also be reduced - need to understand who needs all that detailed information
      • Some was requested by ATLAS and CMS to debug streaming

• SRM+https testing setup
  • Moved under ATLAS for final transfers to and from tape
    • INFN-T1,  already created a test tape endpoint
    • BNLLAKE and BNL-ATLAS will be next
    • Manchester will be used  this specific tests
  • Plan to test all the T1s
  • Important to have data challenges using SRM+https instead of SRM+gsitp