Rucio Meeting
Thursday 12 May 2022, 15:00
→
16:00
Europe/Zurich
4/S-030 (CERN)
4/S-030
CERN
30
Show room on map
Martin Barisits
(
CERN
)
Videoconference
Rucio Development Meeting
Join
Zoom Meeting ID
413496641
Host
Martin Barisits
Alternative hosts
Cedric Serfon, Mario Lassnig, Dimitrios Christidis
Passcode
28849311
Useful links
Join via phone
Zoom URL
15:00
→
15:05
News
5m
Minutes
Ticket review marathon
Martin underway to contact component leads individually to review tickets
15:05
→
15:20
Community News & DevOps roundtable
15m
Minutes
ATLAS
CMS
Issues with conveyor-receiver
Receiver marks requests as DONE while actually in Archiving state
#5532
Fermilab/DUNE/Ikarus
Belle II
DUNE
Scale testing ahead in June
ESCAPE
Jupyterlab extension
New release helped, but still some errors on the GUI
SKAO
RAL/MultiVO
15:20
→
15:50
YugabyteDB (PostgresSQL compatible distributed database)
30m
Speaker
:
Franck Pachot
(
YugabyteDB
)
YugabyteDB - Technical Presentation-2.pdf
15:50
→
15:55
Developers roundtable
5m
Minutes
Progress-2022-05-12.png
Rucio 1.29 "Into the donkeyverse" progress tracking
In Progress
Further ingetragtion of CMS-style consistency checking
#5438
[Igor]
Fix client API documentation page
#5444
[Joel]
Ticket Cleanup Marathon [Martin, All]
Reduce logging of server & daemons
#5449
[Mayank]
script to map raw logs, waiting for MONIT
Still waiting for access to ES
Authentication workflows in WebUI 2.0
#5135
[Mayank]
stuck on setting up base repo for webui
Add python 3.10 tests to CI framework.
#5145
[Mayank]
ongoing, by next week
Difficult to get CC7 to work with py3.10
Setup WebUI repository for external collaboration
#w/12
[Mayank]
Calculation of container length and events
#5422
[Radu]
Difficult to calculate proper statistics for containers without performance degredation
Introduce heartbeat handler to all daemons
#5443
[Radu, ALL]
Started to move some daemons to the new daemon_handler
Further OIDC and SSO optimization and documentation
#5437
[Rizart]
Migration of SQLAlchemy queries to new query syntax (1.4 -> 2.0)
#5247
[Yuyi, ALL]
ongoing (sessions.py)
not yet too scary :-)
Radu: also work on transfer part? how to enable warnings
SetEnv PYTHONWARNINGS always::DeprecationWarning
SetEnv SQLALCHEMY_WARN_20 1
https://docs.sqlalchemy.org/en/14/changelog/migration_20.html
In Review
Hermes 2 evolution
#4610
[Cedric]
mypy static code analysis
#4455
[Mayank, Ben, Joel]
New workflow will test the annotations of new PRs (Amount of un-annotated code should not increase)
PR does not type-check yet, since too many type-annotations are missing
Re-factor/cleanup deletion workflow
#5442
[Radu]
Add session object to API layer
#5439
[Radu]
Done
re-evaluate the greedy = 1TB constant
#5358
[Cedric]
Clean dead code in the probes (ATLAS and common)
#83
[Eric, Dimitrios]
boto vs boto3
#5346
[James, Mario]
Flag transfer errors coming from tape sources
#5410
[Radu]
Remove author list from headers [Martin, Joel]
Todo
Create a contributor guide for probes
#84
[Dimitrios]
Adherence to rule grouping dependent on order of ATTACH
#5251
[Igor]
Create a server/daemon installation howto
#5445
[Joel]
Remove old readthedocs documentation once the new one is fully operational
#5446
[Joel, Martin]
Add symlinks for the new daemon names
#5131
[Joel]
Prepare a clear list of changes which need to be made to use new daemon names
#5132
[Joel]
Reduce core component tickets to less than 10 [Mario, Martin, Gabriele]
Document and cleanup discrepancies between surl and lfn2pfn algorithms
#5129
[Martin]
Unable to Delete File DID via Undertaker
#5154
[Martin]
Removal of Python 2.7 dependencies from the code
#5435
[Martin]
Reduce rule tickets to les than 10 [Martin]
Create developers testing guide in the documentation
#5452
[Mayank, Joel]
Ben working on sub-task to test locally
Remove direct DB dependence from the WebUI
#4319
[Mayank]
Analyze feedback on UI Survet
#w/13
[Mayank]
Further integration of temporary tables to other major workflows
#5441
[Radu]
Remove full_mode switch from conveyor-receiver
#5453
[Radu]
Exchange of function-based indices with normal indices
#5440
[Radu, Mario, Martin]
Re-work/cleanup distance table
#5454
[Radu]
Create documentation page for S3-type storage
#5450
[Rizart]
Exists as GGDoc (
#5284
)
Add support for S3 virtual hosted buckets
#5451
[Rizart]
Test on-premise
Ongoing support issue with AWS/Fresno X.509 shenanigans
Update/Re-design core.meta module
#5224
[Rob]
Merge list_dids and list_dids_extended methods
#5448
[Rob]
Container linux distribution
CC7 images harder to manage (Python 3.10++)
CS8 and CS9 already in testing
-> Separate discussion in one of the next meetings
15:55
→
16:00
AOB
5m