Rucio Meeting

Thursday 20 Jan 2022, 15:00 → 16:00 Europe/Zurich

Martin Barisits (CERN)

15:00 → 15:05 News

• Python 2.7 deprecation meeting
  • Tentatively on February-17, still waiting for feedback
• Google Summer of Code 2022
  • If you are interested to do a Rucio project, please let me know
  • Deadline February 21
  • Two project durations: short (175h), long (350h)
• Testing
  • Please do not "test" your PRs with repeated commit submission on the main rucio repository
  • It takes runner slots for other jobs
  • You can test your development branches on your personal github fork with exactly the same GH actions as the main repository
    • Everything comes out of the box
    • GH Actions need to be turned on

15:05 → 15:20 Community News & DevOps roundtable

• ATLAS
  • Migration of puppet nodes to kubernetes
• CMS
  • Setup cache notification (Rucio cache-consumer) workflow
    • Not quite working
• Fermilab/DUNE/Rubin/...
  • Rubin: Workshop at the moment, Large number of small files coming up
• Belle II
  • Belle II general meeting right now
  • Migration to Python 3 one topic
    • Depending on migration to DIRAC 7.3
  • Moving to IAM
• DUNE/Edinburgh
  • Policy packages in containerized systems
  • bug with policy packages which cause a import loop
• DIRAC
  • PR interface to MultiVO
  • Another bug which seems strange
    • list_scopes syntax not correct? Unexpected keyword argument (Could be filter vs filter_ issue)
    • list_scopes initializes filter with ={}
      •  Probably not a good idea
      • Not recommended (see python documentation)
• RAL/MultiVO
  • Containerized setup not working with tokens
  • S3 RSEs integration
    • No documentation page
• ESCAPE
  • Kick-started discussion with KM3Net
  • CS3Mesh integration with Rucio
    • All the pieces for integration with RIVA should be there

15:20 → 15:25 Hot topics

15:25 → 15:55 Developers roundtable

Progress-2022-01-20.png

• Rucio 1.28 "Teenage Mutant Ninja Donkeys" release roadmap followup
  • IN PROGRESS
    • Fix broken Client and Rest API docs view #4992 [Joel]
      • Decided on framework OpenAPI, now implementing the render
        • Will be a mixture between Py-Doc and FLASK (Flask plugin for OpenAPI)
    • Test framework for VO Specific tests #4797 [Mayank]
      • New test suite: vo-tests
      • Part left:
        • Adding configuration
        • Use composite workflows for repeated workflows (auto-test + vo-tests)
    • Stabilize geoip based replica selection #5133 [Radu]
      • Needs test cases
    • Host python 2.7 deprecation meeting [Martin]
      • Will send out eMails this week
    • Clarify, implement and document a way how to include policy packages in containers/kubernetes #5138 [James]
    • Authentication workflows in WebUI 2.0 #5135 [Mayank]
  • IN REVIEW
    • Hermes 2 evolution #4610 [Cedric]
      • First PR is there, missing documentation
      • Dev container is missing Influx, Elastic, etc. would be needed for TESTING
        • Modify dev container
    • BB8 rewrite #4706 [Cedric]
      • First draft already in December
      • Had discussion with CMS how they want to rebalance data
      • Query to select data is very ATLAS specific, now parameterizable
      • Would be good if CMS has a look on this too
    • Create CMS-style consistency checking daemon #5113 [Stefan]
      • PR is there, including comments from Eric
    • Support prometheus metrics in daemons and servers #3416 [Radu]
  • TODO
    • Host token workflow meeting [Martin]
    • Reduce rule tickets [Martin]
    • Document and cleanup discrepancies between surl and lfn2pfn algorithms #5129 [Martin]
    • Reduce transfer tickets [Radu]
    • Disentangle fts3 specific code from conveyor poller and receiver #4873 [Radu]
    • Add symlinks for the new daemon names #5131 [Joel]
    • Prepare a clear list of changes which need to be made to use new daemon names #5132 [Joel]
    • Clean dead code in the probes (ATLAS and common) #p/83 [Dimitrios, Eric]
    • Create a contributor guide for probes #p/84 [Dimitrios]
    • WebUI 2.0 Architecture #5134 [Mayank]
    • Remove direct DB dependence from the WebUI #4319 [Mayank]
    • Include new operator engine also to json based columns #5144 [Rob]
    • Add MongoDB metadata plugin #5143 [Rob]
  • DONE
    • Create a prototype workflow with temporary tables #5130 [Radu]
    • Write documentation page about helm-charts and versioning #h/84 [Eric]
•  

15:55 → 16:00 AOB