Rucio Meeting

Thursday 29 Jun 2023, 15:00 → 16:00 Europe/Zurich

Martin Barisits (CERN)

15:00 → 15:05 News

• Rucio 32 schedule
  • Code-Freeze on Jul 24
  • 32.0.0rc1 on Jul 26
  • 32.0.0 (final) on Jul 31
• Summer meeting schedule
  • Bi-weekly meeting schedule in July & August!
  • Jul 6
  • Jul 20
  • Aug 3
  • Aug 17
  • Aug 31
  • September back to weekly schedule
• Found a rule evaluation issue attaching datasets/containers to containers #6234
  • Rue eval not triggered if use_temp_tables is set to True
  • This issue also existed previously for the non-temp table workflows, but was fixed in July-2022 via #5676
  • A fix will go into next weeks release, but it might be advisable to check rules on containers which might be prone to this issue
    • e.g. datasets/containers added to a container after the replication rule on the was created

15:05 → 15:25 Community News & DevOps roundtable

• ATLAS
  • Eval bug mentioned before
    • Will share more efficient way, if possible, to identify affected rules/containers
• CMS
  • Issue with updated_col_rep script
    • Repeatedly running into the issue that the job cannot execute anymore, because the queue is just too big
      • Optimized procedure which splits the workspace can be used
    • account_usage with negative numbers
      • Martin can sent script to reset everything to current state
      • Update can be run as procedure or via daemons
• Fermilab DUNE/co
• RAL/MultiVO

15:25 → 15:45 ALTO plans

Speaker: Y. Richard Yang

2023-06-29-rucio-weekly.pdf

2023-06-29-rucio-weekly.pptx

15:45 → 15:55 Developers roundtable

Progress-2023-06-29.png

• 32 "The Good, The Bad and the Donkey" Priority followup
  • In Progress
    • Support protocols with NULL priority #6201 [Dimitrios]
    • Incorporate IAM-based testing for token workflows #6188 [Eraldo, Dimitrios]
      • Basic config from IAM team at CERN
    • Kick off SIG-Tokens [Martin, Dimitrios]
    • Migrate Dashboard to Clean Architecture #w/158 [Mayank, Anton]
    • Internal transfer monitoring #6189 [Radu]
      • Refactor to a full graph view of the transfers
    • New dispatcher logic for conveyors #6190 [Radu]
      • Prototype quite compliacted (Relies on async python)
      • Need to assess if the complexity is worth the benefit
  • In Review
    • Migrate DID and DID-metadata view to clean architecture #w/160 [Mayank, Anton]
    • Migrate List My Rules and Rule View to clean architecture #w/159 [Mayank, Anton]
    • Create Pages for Create Rule Workflow #w/144 [Anton]
    • Unable to Delete File DID via Undertaker #5154 [Anton]
    • Adapt probes to rucio/rucio #5804 #p/108 [Eric]
      • Started running in ATLAS this morning
      • Eric already adressed comments
  • Todo
    • Design and implement a token request module #6187 [Dimitrios]
    • Design and implement a token cache module #6183 [Dimitrios]
    • Reduce core tickets <15 [Martin, Mario]
    • foreign key error on deleting dids in reaper #5733 [Martin]
    • Support OIDC login #w/149 [Mayank]
    • Exchange of function-based indices with normal index #6143 [Radu]
  • Done
    • Replace all instances of X509 identities presented in the legacy format #6182 [Dimitrios]
  • Delayed
    • Update/Re-design core.meta module #5224 [Rob]
    • Document and cleanup discrepancies between surl and lfn2pfn algorithms #5129 [Martin]
    • Create a server/daemon installation howto #d/178 [Mayank]
    • Create developers testing guide in the documentation #d/177 [Mayank]
• Documentation corner
  • Document environmental variables affecting the client #171 [Dimitrios]
  • Documentation and dev guidelines for MyPi type annotations #116 [Mayank, Martin]
  • Create developers testing guide in the documentation #177 [Mayank]
  • Create a server/daemon installation howto #178 [Mayank]
    • Ubuntu-specific PR #215
  • Improve documentation on rucio.cfg vs configuration table #183 [Radu]
  • Add an FAQ-style entry aimed at users for STUCK rules #184 [Fabio]
  • Add instruction about DB partitioning #185 [Martin]
  • bridge the gap between running rucio in demo env and full production deployment #187 [Radu]
  • Introduce documentation on subscriptions #190 [Cedric]
  • Page about the different event types and messages (ActiveMQ) #226 [Eraldo]
• Other topics
  •  

15:55 → 16:00 AOB