Skip to main content
 
Focus on:  Hide Contributions
Login

Rucio Meeting

→ Europe/Zurich
4/S-030 (CERN)

4/S-030

CERN

30
Show room on map
Martin Barisits (CERN)
Rucio Development Meeting
Join
    • 15:00 → 15:05
      News 5m
      • July & August meeting schedule
        • Bi-weekly meetings over the summer again!
        • July 7
        • July 21
        • August 4
        • August 18
        • September 1 (Back to weekly schedule)
      • Rucio workshop 2022
        • Will be co-located with the WLCG Workshop
          • Lancaster, UK
          • Nov 7-11
            • Most likely first 3 days WLCG workshop, last 2 days Rucio workshop
        • Program committee still needs to be established, will start work soon
        • Official announcement eMail to come soon
      • Rucio 1.29 "Into the Donkeyverse"
        • Code-freeze since this morning!
        • 1.29.0rc1 tomorrow or Monday
          • Depending on Merge progress
    • 15:05 → 15:20
      Community News & DevOps roundtable 15m
      • ATLAS
        • RPG development
          • RPG is a system in ATLAS managing some data flows
          • Idea is to get RPG functionality into Rucio Rules & Subscriptions
          • #4508
      • CMS
      • Fermilab/Ikarus/DUNE/Rubin
        • Getting DUNE policy package (lfn2pfn method) to work
        • FTS3 testing for Rubin
          • Tuning transfer parameters
        • Setting up cluster deployment for Rubin
      • DUNE
        • Testing policy package for lfn2pfn method
          • Ahead of data challenge in July
          • Some issues, working on resolving these
      • Belle II
        • Testing 1.28.5 release (Running 1.26 in production)
        • Migration to 1.29 still needs DIRAC python3 deployment
      • ESCAPE
        • Issues with replication, some transfers take too long, needs investigation
          • Might be an issue with large conveyor-submitter bulk_size
      • SKAO
        • Transition from user_token to admin_token
          • Identified quite a few problems 
            • Some Rucio related, some otherwise (FTS, ...) 
            • One bug in reaper, conveyor-poller needs to be updated too
          • No group claim ('wlcg.groups') existent in token
            • Some storages might require the claim -> doesn't work
      • RAL/MultiVO
        • K8s deployment
          • Daemons deployed
          • Server & Auth server - x509 auth does not work yet
          • proxy renewal cron needs a bit of work to renew multiple proxies for multi-vo
    • 15:20 → 15:25
      Hot topics 5m
    • 15:25 → 15:55
      Developers roundtable 30m
      • Rucio 1.29 "Into the donkeyverse" progress tracking
        • In Progress​​
          • Further ingetragtion of CMS-style consistency checking #5438 [Igor]
            • Waiting for PR to be merged - then this is done (until further steps are identified)
          • Remove old readthedocs documentation once the new one is fully operational #5446 [Joel, Martin]
            •  
        • In Review
          • Hermes 2 evolution #4610 [Cedric]
            • Comments to be included
          • Calculation of container length and events #5422 [Radu]
            • Difficult to calculate proper statistics for containers without performance degradation
            • Open PR, but unclear if it fully closes the issue
          • Further OIDC and SSO optimization and documentation #5437 [Rizart]
            • Ticket done, now looking on related ticket which will be submitted as one
          • Removal of Python 2.7 dependencies from the code #5435 [Martin, Joel]
          • Further integration of temporary tables to other major workflows #5441 [Radu]
        • Done
          • re-evaluate the greedy = 1TB constant #5358 [Cedric]
          • Clean dead code in the probes (ATLAS and common) #83 [Eric, Dimitrios]
          • boto vs boto3 #5346 [James, Mario]
          • Fix client API documentation page #5444 [Joel]
          • Ticket Cleanup Marathon [Martin, All]
          • mypy static code analysis #4455 [Mayank, Ben, Joel]
            • Discussion for type checking and documentation for developers guide coming
          • Analyze feedback on UI Survey #w/13 [Mayank]
          • Setup WebUI repository for external collaboration #w/12 [Mayank]
          • Flag transfer errors coming from tape sources #5410 [Radu]
          • Re-factor/cleanup deletion workflow #5442 [Radu]
          • Add session object to API layer #5439 [Radu]
          • Remove author list from headers [Martin, Joel]
        • Todo
          • Create a contributor guide for probes #84 [Dimitrios]
          • Adherence to rule grouping dependent on order of ATTACH #5251 [Igor]
          • Document and cleanup discrepancies between surl and lfn2pfn algorithms #5129 [Martin]
          • Unable to Delete File DID via Undertaker #5154 [Martin]
          • Exchange of function-based indices with normal indices #5440 [Radu, Mario, Martin]
          • Create documentation page for S3-type storage #5450 [Rizart]
            • Exists as GGDoc (#5284)
          • Add support for S3 virtual hosted buckets #5451 [Rizart]
            • Test on-premise
            • Ongoing support issue with AWS/Fresno X.509 shenanigans
          • Merge list_dids and list_dids_extended methods #5448 [Rob]
        • Delayed
          • Update/Re-design core.meta module #5224 [Rob]
          • Reduce rule tickets to les than 10 [Martin]
          • Reduce core component tickets to less than 10 [Mario, Martin, Gabriele]
          • Create a server/daemon installation howto #5445 [Joel]
          • Add symlinks for the new daemon names #5131 [Joel]
            • Prepare a clear list of changes which need to be made to use new daemon names #5132 [Joel]
          • Migration of SQLAlchemy queries to new query syntax (1.4 -> 2.0) #5247 [Yuyi, ALL]
          • Create developers testing guide in the documentation #5452 [Mayank, Joel]
            • Ben working on sub-task to test locally
          • Remove direct DB dependence from the WebUI #4319 [Mayank]
          • Reduce logging of server & daemons #5449 [Mayank]
            • Now access to ES
            • Started to populate dashboards about frequency distribution of log patterns
            • --> DELAYED for codefreeze
          • Authentication workflows in WebUI 2.0 #5135 [Mayank]
            • stuck on setting up base repo for webui
            • --> together with GSoC student, DELAYED for codefreeze
          • Add python 3.10 tests to CI framework. #5145 [Mayank]
            • Fedora images are ready for 3.10
            • Stopped to work on CC7 - not worth it
            • Currently looking into centos stream 8 and stream 9
            • --> On track
          • Remove full_mode switch from conveyor-receiver #5453 [Radu]
          • Re-work/cleanup distance table #5454 [Radu]
          • Introduce heartbeat handler to all daemons #5443 [Radu, ALL]
            • Started to move some daemons to the new daemon_handler
      • Other topics
        •  
    • 15:55 → 16:00
      AOB 5m
Powered by Indico v3.3.7-pre