BiLD (Bi-weekly DIRAC Development meeting) – 06/05/2021

At CERN: Still, nobody :/
On Zoom: Federico, Alexandre, André, Andrei, Andrii, Cedric, Christophe, Christopher, Daniela, Hideki, Janusz, Marko, Simon, Xiaomei
Apologies:

Follow-up from previous meeting

• Ran hackathon last week on v7r3-pre7:
  • trello board here
  • VMDIRAC: tag was not there, so the latest version was not installed.
    • Q: who creates the tags?
      • Simon will create the tags
  • ComponentMonitoring still looks problematic.
• PRs merging strategy?
  • Marko Working on GitHub Action – should be finished tomorrow
    • prob only applying from v7r2

DIRAC communities roundtable

GridPP:

Daniela

• Overload of the file catalog follow-up?
  • haven’t seen it again (no changes done)
• NeTR

CLIC:

André

• NTR

LHCb:

Federico+Chris+Chris+Marko

• Nothing interesting to report

EGI (+ France Grilles):

Andrei

• M2Crypto and VOMS extension follow-up (issue)
  • operationally solved for the moment
  • reproducer is needed for trying to fix the issue – will be added to issue
• problem with retrieving the output for ARC
  • v7r1 is using DIRACOS v1r18 so no arc6 (which is in v7r2)
  • will try to install it with --dirac-os-version=v1r21
• Moved all components from cyfronet, migration is over

Belle2

Hideki

• v7r0 migration – ongoing
• updated DIRACOS to v1r18 (in v6r22)
  • webdav now working fine

BES3+JUNO

Xiaomei

• NTR

DIRAC releases

• v7r1p39:
  • (#5121) for HTCondor, the SiteDirectory write the executable in the globally defined working directory
• *v7r2p6:
  • (#5102) JobCleaningAgent will first DELETE and only then REMOVE jobs
• *v7r3-pre7:
  • (#5110) Glue2 is now the only way
  • (#5000) Initial support for Python 3 server installations

DIRAC projects

DIRAC:

Issues by milestone:

• v7r1:
  • Some recently added issues
  • Some old ones for documentation still to finish
• v7r2:
  • NTR
• v7r3:
  • NTR

PRs discussed:

• [v7r1] Add sanity check for reading CA dir
  • Christophe do we need this check at all?
  • Daniela I think it might be useful, but it’s a corner case. In theory it should never happen
  • finally, remove the check
• [v7r1] Add multi-node allocation support in the CEs (srun with LocalCE and SSHCE)
  • for HPC (for executing multiple pilots in parallel)
  • Logic looks fine, but needs another review
• [v7r2] Make DIRAC.Core.Base.Client.Client threadsafe
  • Looks OK but needs review
• [v7r3] register args (optional)
  • this is OK and can be merged. [v7r3] move Script goods to DIRACScript(wait #5061) will need the above merged and some more refactoring after that

WebApp:

• Preparing for pip package: created release v4r3-pre3
  • ToDo:
    • change for adapating to new structure
    • upload to pypi
• couple fixes after hackathon merged and released

Pilot3:

• NTR

DIRACOS:

• xroot5?
  • NTR
• arc6
  • Looks fine: in v1r21 (from v7r2p6)
• condor 8.8?
  • NTR

DIRACOS2:

• from previous meetingfts-rest on py3?
  • NTR
• from previous meeting Question for rucio client… (issue)?
  • Not done yet (not fully in our hands)
• NeTR

VMDIRAC:

• NTR

Documentation:

• NTR

OAuth2:

• from previous meeting Any news from issue WLCG and Token transition – reminders and requirements ?

  • Recently had a discussion with IaM developer (Andrea). Mostly technical.

• We created a second certification setup where to test these OAuth interactions: CC7 box is there, as well as MySQL 8: started installing DIRAC there. The dteam IaM instance will be added there.

  • it was discussed if to add also the LHCb IaM but we decided against it
  • we should probably need admin access to dteam IaM – probably also managed by Andrea
    • Andrei+Andrii will ask about that

• Andrii forwarded https://docs.google.com/document/d/1ZQzElD866yV6t9AomeW6r-L9s-64pNB95OFDd1EZEbA/edit?ts=60814a09#heading=h.ur0csvxkk8tc which contains technical drawings about the proposed implementation.

  • anyone can comment on it as now. Please do read it!

• Andrii I will split the existing PR for making it easier to comment/understand

tornado/HTTPs

• NTR

management

• from previous meeting 1 task left for deploying on CVMFS
  • Not yet done
• Add same check to VOMS_DIR and VOMSES as there is for CERT
  • Asked to add the created bashrc to qa branch of puppet modules
• Q: Test on CentOS8? (for DIRACOS2/python3)
  • not really any need to do them (being conda-based it should be OK)

COMDIRAC

• NTR

other externals, including Rucio

• the Rucio client as of now requires a configuration file
  • changes there does not depend from us: issue has been opened and it’s being looked into.
• PR: https://github.com/DIRACGrid/DIRAC/pull/5067
  • failing tests
  • no updates since last time
  • Cedric I would like to test the changes better that now. E.g. using github actions, or in a certification setup
    • Federico For the moment (this PR is for v7r0) just make it to pass the existing tests, and we can merge it. Once merged, the integration tests run by GitHub Actions can be added to release branch v7r2.

Release planning, tests and certification

• Certification machine(s)

  • created wiki: https://github.com/DIRACGrid/DIRAC/wiki/Certifications where there are some reminders + monitoring links for the 2 certification machines.

• Next hackathon

  • in 2 weeks (20th May)
    • “regular” v7r3-preX

Weekly development(s) focus

NTR

AOB and topics from Google forum

• DUW next week
  • Timetable is ready: https://indico.cern.ch/event/852597/timetable/#all.detailed

Next hackathon on May 20th: https://indico.cern.ch/event/1037152
Next BiLD on June 03rd

LHCbDIRAC

• v10r2 planning in https://gitlab.cern.ch/lhcb-dirac/LHCbDIRAC/-/wikis/Releases-planning#based-on-dirac-v7r2
• https://codimd.web.cern.ch/2es8HPUcSmS42jrTxhX3JQ?both
  • No news
• At some points we should install a python3 client on CVMFS
  • in the hands of lb-dirac (then LHCbPilot would need to add a flag)