Rucio Development Meeting

Thursday Jun 10, 2021, 3:00 PM → 4:00 PM Europe/Zurich

Martin Barisits (CERN)

3:00 PM → 3:05 PM News

• Code freeze Jun-22
  • 1.26.0rc1 Jun-24
  • 1.26.0 final Jul-2
• CHEP Rucio Paper
• Docker Auto Build
• Next week no Dev meeting!

3:05 PM → 3:15 PM Community News & DevOps roundtable

• ATLAS
  • All working well
  • SFO Tests ongoing at the moment
  • 1.25.6 on INT
• CMS
• Fermilab
  • Adapting openshift deployment to 1.25* (py3 issues)
• Belle II
  • Metadata
    • Query metadata for certain did type -> full table scan
    • Solution
      • Add index on did_type for json table
      • Data needs to be synced though for existing did_meta rows
• LDMX
  • Q: How do non-CERN VOs get robot certificates?
    • Belle II: Ask Ueda :-)
• ESCAPE
  • Amazon S3 cloud storage config
    • Mario supplied some config, needs to be added on Rucio side
• SKAO

3:15 PM → 3:25 PM Hot topics

• Puppet Manifests
  • Belle II interested in centrally supplied puppet manifests
  • ATLAS puppet manifests
    • Extremely convoluted and CERN/ATLAS specific
    • Substantial work to make commonly usable
  • Collecting/providing central puppet manifests (on rucio repo) possible, but needs to be driven by a Community
  • Cedric will check with Belle II
• Signing docker images
  • Would be useful due to security rules of some organisations
  • Need to check workflows
• Metadata
  • multiple communities looking into generic metadata
  • How to best join forces?
    • Slack channel to coordinate
  • Belle II,  ESCAPE, SKAO, interested
  • Inequality development definitely needed
• Docker Auto builds
  • Can ask for OS waiver at the moment
  • If we want to go the route of signing docker images, would be better to do image creation, signing and push from within cern
• New requirements model
  • Ben presented new requirements model

3:25 PM → 3:55 PM Developers roundtable

Rucio 1.26 "Donkey League of La Mancha" priority followup

• In Progress
  • Test and switch to new rule algorithm #4215 [Martin]
    • Remove old code for 1.26
  • Re-Factor Transfer workflow [Radu]
    • PR updated with all the comments
    • Review needed
  • Discuss and plan mysql5 deprecation [Mario]
    • Removal of testcases just being submitted
  • Test SSO Login [Rizart]
    • Studied API and look into connection to CERN SSO
    • Checking with CRIC since they already successfully did this
    • Application registered on CERN-SSO
  • Hermes2 planning
    • Add index for services column #4647
      • Done, needs merging
  • Setup.py and requirements overhaul #4456 [Ben]
    • Done
      • Needs merging
  • Versioned history tables #2063 [Martin]
    • Ongoing
• Todo
  • Switch to policy-packages as default [James, Martin]
  • Cleanup tickets #4463 [*]
• Done
  • Remove webpy endpoints and dependency #4044 [Ben]
  • Migrate documentation to new docusaurus [Martin, Ben, Daniel]
  • Rename reaper2 to reaper #4462 [Martin]
  • Test Import and Export of CRIC JSON [Rizart]
  • Python 3.8 and 3.9 testing #4453 [Ben]
• Delayed
  • QoS #3419 [Mario, Martin, Rizart]
  • Auditor overhaul #3437 [Dimitrios, Eric]
  • rucio.cfg vs Config table #2630 [Mario]
  • Logging Review #4220 [All Component leads]

Developer roundtable

• TPC 
  • Done and merged
• WebUI updates
  • Lock query update to include request state
    • Will need join between locks and requests table

3:55 PM → 4:00 PM AOB