Rucio Meeting

Thursday 13 Jan 2022, 15:00 → 16:00 Europe/Zurich

Martin Barisits (CERN)

15:00 → 15:05 News

• Happy new year everyone!
  • Back to weekly (remote) meetings
• Component leadership update
  • Igor joined Dimitrios for Consistency Checking!
    • Thanks!
• January release schedule
  • 1.23.18 LTS Jan-10
    • Moved to Feb-07 due to no changes
  • 1.27.3 Jan-17
  • 1.26.11 LTS Jan-24
  • 1.27.4 Jan-31
  • (1.28 "Teenage Mutant Ninja Donkeys" March)
• Conferences 2022
  • CHEP moved to May 2023
  • ACAT Oct
  • Please let me know if you are interested in common Rucio submissions
• Extended/Topical Rucio meetings
  • Python 2.7 deprecation
  • Token workflow evolution in Rucio
  • I would like to host both in Q1-2022
    • Needs community attendance to be useful
    • Probably in a Rucio meeting timeslot, but extended (90m/120m?)

15:05 → 15:20 Community News & DevOps roundtable

• ATLAS
  • Monitoring/Database hickup on NYE, otherwise ran very stable
• CMS
  • Mostly smooth
  • Recovered from the Multihop issue without timeouts set (Replicas stayed around then)
  • Tape-overwrite
    • Conflicting reports wether it is working or not (Declaring tape replica as OK if file is on tape and checksum matches)
      • Not clear if there is a problem, needs more monitoring
• Fermilab/DUNE/...
  • Mostly smooth
  • Deleting a DID, Undertaker processes it, but nothing happens
    • Comment: For files the undertaker removes the expiration and nothing happens, since originally this was only meant for datasets
    • Ticket to follow up
  • Workshop for Rubin next week on Testing
• Belle II
  • B2 also interested in Tape-Overwrite
    • Probably requires FTS 3.11 
• DUNE/Edinburgh
  • Finished work on client-side policy packages
  • Next: How to best deploy policy packages in containers/kubernetes driven environemnts
    • James will contact people deploying via containers
• MultiVO/RAL/STFC
  • Worked well over Christmas
  • Now working on Token Authentication
    • Accounts in IAM, but login does not succeed (yet)
  • Summer student at RAL to work on Rucio for MultiVO
    • Please contact Tim if you have any ideas
• ESCAPE
  • CS3Mesh discussion about Tokens for REVA

15:20 → 15:25 Hot topics

15:25 → 15:55 Developers roundtable

Progress-2022-01-13.png

• Rucio 1.28 "Teenage Mutant Ninja Donkeys" release roadmap followup
  • IN PROGRESS
    • Fix broken Client and Rest API docs view #4992 [Joel]
      • Decided on framework OpenAPI, now implementing the render
        • Will be a mixture between Py-Doc and FLASK (Flask plugin for OpenAPI)
    • Test framework for VO Specific tests #4797 [Mayank]
    • Disentangle fts3 specific code from conveyor poller and receiver #4873 [Radu]
    • Add: Prometheus to release roadmap
  • IN REVIEW
    • Hermes 2 evolution #4610 [Cedric]
      • First PR is there, missing documentation
      • Dev container is missing Influx, Elastic, etc. would be needed for TESTING
    • Create a prototype workflow with temporary tables #5130 [Radu]
      • Opened PR for it
      • Has configuration option to activate the temporary table workflow
      • Uses private temporary tables
    • BB8 rewrite #4706 [Cedric]
      • First draft already in December
      • Had discussion with CMS how they want to rebalance data
      • Query to select data is very ATLAS specific, now parameterizable
      • Would be good if CMS has a look on this too
    • Create CMS-style consistency checking daemon #5113 [Stefan]
      • PR is there, including comments
  • TODO
    • Host python 2.7 deprecation meeting [Martin]
    • Host token workflow meeting [Martin]
    • Reduce transfer tickets [Radu]
    • Reduce rule tickets [Martin]
    • Stabilize geoip based replica selection #5133 [Radu]
    • Document and cleanup discrepancies between surl and lfn2pfn algorithms #5129 [Martin]
    • Add symlinks for the new daemon names #5131 [Joel]
    • Prepare a clear list of changes which need to be made to use new daemon names #5132 [Joel]
    • Clarify, implement and document a way how to include policy packages in containers/kubernetes #5138 [James]
    • Clean dead code in the probes (ATLAS and common) #p/83 [Dimitrios, Eric]
    • Create a contributor guide for probes #p/84 [Dimitrios]
    • Authentication workflows in WebUI 2.0 #5135 [Mayank]
    • WebUI 2.0 Architecture #5134 [Mayank]
    • Remove direct DB dependence from the WebUI #4319 [Mayank]
    • Include new operator engine also to json based columns #5144 [Rob]
    • Add MongoDB metadata plugin #5143 [Rob]
  • DONE
    • Write documentation page about helm-charts and versioning #h/84 [Eric]

15:55 → 16:00 AOB