Name: Rucio Meeting
Start: 2022-06-09T15:00:00+02:00
End: 2022-06-09T16:00:00+02:00
Location: CERN

1

News

1.29 code freeze on Jun 30!
Next week no meeting
Jun 23 SIG Metadata Status

2

Community News & DevOps roundtable

ATLAS
- Several enhancements planned for subscriptions for ATLAS data flow improval
- Waiting for davix update on FTS (For Amazon transfers)
CMS
- File DIDs being deleted for unknown reasons
  - Why does undertaker delete files?
- Since 1.28 upgrade Kronos issues
  - Lots of connections
  - AMQ service managers complain (Not an issue for ATLAS, probably because we do not use SSL with username/password)
  - Possible an issue with stomp.py version (We use 6.1.1 which is yanked)
Rubin
- How to manage configuration in the collaboration
Icarus
- Possibly some issues with reaper
DUNE
- Launch data challenge next week
  - At the level of expected proto-dune data taking
- Testing policy package code
Belle II
- Fixing unit tests for Belle II convention
  - Have all tests run for Belle II
  - Some methods identified which won't work with schemas with `/` in did names
    - Needs to be made compatible
- Py3 by end of summer
ESCAPE / EOSC-future
- Renewing certificates
SKAO
- Upgrading to 1.28.4 for reaper oidc functionality

3

Static type checking and type annotations

Speakers: Joel Dierkes (University of Magdeburg (DE)), Mayank Sharma (University of Texas at Arlington (US))

4

Developers roundtable

Rucio 1.29 "Into the donkeyverse" progress tracking
- In Progress
  - Further ingetragtion of CMS-style consistency checking #5438 [Igor]
    - Waiting for PR to be merged - then this is done (until further steps are identified)
  - Remove old readthedocs documentation once the new one is fully operational #5446 [Joel, Martin]
  - Create a server/daemon installation howto #5445 [Joel]
  - Add symlinks for the new daemon names #5131 [Joel]
    - Prepare a clear list of changes which need to be made to use new daemon names #5132 [Joel]
  - Reduce logging of server & daemons #5449 [Mayank]
    - Now access to ES
    - Started to populate dashboards about frequency distribution of log patterns
  - Authentication workflows in WebUI 2.0 #5135 [Mayank]
    - stuck on setting up base repo for webui
  - Add python 3.10 tests to CI framework. #5145 [Mayank]
    - Fedora images are ready for 3.10
    - Stopped to work on CC7 - not worth it
    - Currently looking into centos stream 8 and stream 9
  - Introduce heartbeat handler to all daemons #5443 [Radu, ALL]
    - Started to move some daemons to the new daemon_handler
- In Review
  - Hermes 2 evolution #4610 [Cedric]
    - Comments to be included
  - Calculation of container length and events #5422 [Radu]
    - Difficult to calculate proper statistics for containers without performance degradation
    - Open PR, but unclear if it fully closes the issue
  - Further OIDC and SSO optimization and documentation #5437 [Rizart]
    - Ticket done, now looking on related ticket which will be submitted as one
- Done
  - re-evaluate the greedy = 1TB constant #5358 [Cedric]
  - Clean dead code in the probes (ATLAS and common) #83 [Eric, Dimitrios]
  - boto vs boto3 #5346 [James, Mario]
  - Fix client API documentation page #5444 [Joel]
  - Ticket Cleanup Marathon [Martin, All]
  - mypy static code analysis #4455 [Mayank, Ben, Joel]
    - Discussion for type checking and documentation for developers guide coming
  - Analyze feedback on UI Survey #w/13 [Mayank]
  - Setup WebUI repository for external collaboration #w/12 [Mayank]
  - Flag transfer errors coming from tape sources #5410 [Radu]
  - Re-factor/cleanup deletion workflow #5442 [Radu]
  - Add session object to API layer #5439 [Radu]
  - Remove author list from headers [Martin, Joel]
- Todo
  - Create a contributor guide for probes #84 [Dimitrios]
  - Adherence to rule grouping dependent on order of ATTACH #5251 [Igor]
  - Document and cleanup discrepancies between surl and lfn2pfn algorithms #5129 [Martin]
  - Unable to Delete File DID via Undertaker #5154 [Martin]
  - Removal of Python 2.7 dependencies from the code #5435 [Martin, Joel]
  - Exchange of function-based indices with normal indices #5440 [Radu, Mario, Martin]
  - Create documentation page for S3-type storage #5450 [Rizart]
    - Exists as GGDoc (#5284)
  - Add support for S3 virtual hosted buckets #5451 [Rizart]
    - Test on-premise
    - Ongoing support issue with AWS/Fresno X.509 shenanigans
  - Merge list_dids and list_dids_extended methods #5448 [Rob]
- Delayed
  - Update/Re-design core.meta module #5224 [Rob]
  - Reduce rule tickets to les than 10 [Martin]
  - Reduce core component tickets to less than 10 [Mario, Martin, Gabriele]
  - Further integration of temporary tables to other major workflows #5441 [Radu]
  - Migration of SQLAlchemy queries to new query syntax (1.4 -> 2.0) #5247 [Yuyi, ALL]
    - ongoing (sessions.py)
    - not yet too scary :-)
    - Radu: also work on transfer part? how to enable warnings
      - SetEnv PYTHONWARNINGS always::DeprecationWarning
        SetEnv SQLALCHEMY_WARN_20 1
        https://docs.sqlalchemy.org/en/14/changelog/migration_20.html
    - Now trying to build a test to identify missing queries
  - Create developers testing guide in the documentation #5452 [Mayank, Joel]
    - Ben working on sub-task to test locally
  - Remove direct DB dependence from the WebUI #4319 [Mayank]
  - Remove full_mode switch from conveyor-receiver #5453 [Radu]
  - Re-work/cleanup distance table #5454 [Radu]

5

AOB

Rucio Meeting

4/S-030

CERN