Rucio Development Meeting

Thursday 18 Oct 2018, 15:00 → 16:30 Europe/Zurich

3196/R-023 (CERN)

Martin Barisits (CERN)

1 News

• Rucio Coding Camp
  • Doodle
  • Preferred week is 26-30 Nov
    • What would be the best days? Thursday+Friday?
      • → Thursday+Friday would be best
• Rucio Community Workshop 2019
  • Call for hosting open until tomorrow!
• Rucio 1.18.4 release on Friday 26th instead of Monday 29th

2 News from the experiments

• ATLAS
  • Network test
    • 30TB CERN - SARA_MATRIX
    • In general a success, some findings
    • Manual configuration of network
    • For the future, it might be beneficial if e.g. Rucio decides some of these parameters dynamically
• CMS
  • Good progress with setup at CERN
    • Timeframe for oracle11 db is 2-3 weeks
      • Martin and Gancho currently re-working the Oracle schema.sql (on github); This could be used to initialize DB as it has some oracle-specific adaptions.
• Belle II
  • Rucio presentation at BNL early Nov

3 Hot topics

• New version of urllib3 (1.24) not compatible with requests 2.19.1
  • Dependency graph not resolved when you do pip install -r
  • Had to pin urllib3 to 1.23
  • This apparently also affects existing PyPi packages
    • Unclear why this affects pip installed package, as normally pip should resolve the dependency graph
    • To be investigated → #1692

4 Developers roundtable

• Python 3 Compatibility
  • Since #1505 there is a Travis test to ensure that a previously Python3 compatible module does not loose compatibility
  • Compatible files are tagged as "# PY3K COMPATIBLE" in the header
  • Let's make an effort to make our modules compatible!
• TFC rules for protocols #1543
  • Plan is to put them into the extended_attributes field in protocol
  • Currently the field is 1048 CHARS
  • Change to CLOB or JSON type
  • Discussion: It's probably best to put it in extended_attributes; So we will go ahead with #1543
• Archive support (Overview ticket #1091)
  • Client needs to extract constituent from archive #1354
    • Dataset information needs to be added to list_replicas #1677
      • Mario on it
  • Bug with checksums in list_replicas return #1613
  • Multiple tickets about handling archives in reaper #1431
    • Need volunteer for these!
• Minimize calls in downloadclient
  • Call list_replicas in downloadclient only once #1427
    • This is related to #1677 as well
  • rucio download should be able to use locally cached metalink #1415
  • Downgrading these tickets from High Priority, Archive tickets should go first
• Change list-dids to type collection, only ALL if --filter is given  #1422
  • Should be fine to change
  • Let's change it back!
    • Add to the help that standard behaviour is only Collection, Do this for files...
• Raise an exception in case of server timeout #1152
  • Possibly very difficult to do as this might be already outside the wsgi container;
  • Who wants to investigate this?
    • Martin will look into it and volunteer some help!
• 404 issue #1568
  • Any news?
    • Could not reproduce with curl; Reproducable with arcls
    • Mario looking into it, needs more time;

5 AOB

• Next week
  • Rucio 1.19.0 "Fantastic Donkey" release planning (medium term plan)
  • This is the place to discuss Ideas and propose developments!