BiLD-Dev

→ Europe/Zurich
Federico Stagni (Conseil Europeen Recherche Nucl. (CERN)-Unknown-Unknown)
Description

Bi-Weekly "Loyal" DIRAC developers meeting. And, following, the LHCbDIRAC developers meeting.

Join Zoom Meeting
https://cern.zoom.us/j/91083694183?pwd=ZkdDY1I1YkJVc2o3UTdBY1BRZE15UT09

Meeting ID: 910 8369 4183
Passcode: 12345678
One tap mobile
+41432107108,,91083694183# Switzerland
+41315280988,,91083694183# Switzerland

Dial by your location
        +41 43 210 71 08 Switzerland
        +41 31 528 09 88 Switzerland
        +41 43 210 70 42 Switzerland
        +33 1 8699 5831 France
        +33 1 7037 2246 France
        +33 1 7037 9729 France
Meeting ID: 910 8369 4183
Find your local number: https://cern.zoom.us/u/artUfAdNB

Join by SIP
91083694183@188.184.89.188
91083694183@188.185.118.153

Join by H.323
188.184.89.188
188.185.118.153
Meeting ID: 910 8369 4183
Passcode: 12345678

BiLD (Bi-weekly DIRAC Development meeting) – 11/03/2021

At CERN: Still, nobody :/
On Zoom: Federico, Alexandre, André, Andrei, Andrii, Cedric, Christophe, Christopher, Daniela, Hideki, Janusz, Marko, Simon, Vladimir, Xiaomei
Apologies:

Follow-up from previous meeting

  • Ran hackathon last week on v7r2-pre37:

    • trello board here
    • VMDIRAC not yet fully installed nor tried
    • ran with CS on https, minor fixes had to be done (mostly doc). Also to puppet module
      • v7r1 clients are not compatible with installations where the CS runs on https, but it was expected
    • Some more fixes had to be done
  • PRs merging strategy?

    •  André No time to investigate yet.
    •  Andrei I made some remarks, I’d like an update. If we are switching to this mode, we never come back.
      •  Christopher maybe not too bad. We can go back to old mode if we don’t like it.
      •  Andrei we can’t anyway keep both in parallel, it’s one or the other.
    • “Random” developers need special attention.

DIRAC communities roundtable

GridPP:

 Daniela

  • NTR

CLIC:

 André

  • Scheduling update to v7r0

LHCb:

 Federico+Chris+Chris+Marko

  • We ran yesterday a LHCbDIRAC hackathon with v7r2-pre38. Quite OK, a few PRs with minor fixes created.
    • Managed to run a few services on a kubernetes cluster.
  • Using CTA at CERN (replacement of Castor)
    • CTA will also be used at RAL

EGI:

 Andrei

  • Still preparing the migration.

France Grilles:

 Andrei

  • Started running some SiteDirectors with Pilot3

Belle2

 Hideki

  • NTR

BES3/Juno:

 Xiaomei

  • Still preparing update to v7r0.

DIRAC releases

  • v7r1p31 + v7r1p32:
    • CHANGE: (#5001) JobAgent: if we don’t match a job, independently from the reason, we wait a bit longer before trying again
    • CHANGE: (#5001) JobAgent: moved check of availability of CE before doing anything else
    • NEW: (#4998) JobWrapper - find executables from PATH if not using an absolute filename
  • v7r2-pre37 + v7r2-pre38:
    • CHANGE: (#4997) Use prompt-toolkit for password input when generating proxies
    • CHANGE: (#4994) dirac-install.py should now be taken from management
    • CHANGE: (#5010) Use ConfigurationClient in CSCLI and CSShellCLI
    • CHANGE: (#5026) Remove DIRAC.Core.Utilities.ExitCallback
    • CHANGE: (#5019) Remove IRODSStorageElementHandler

DIRAC projects

DIRAC:

By milestone:

  • v7r1:
    • almost only documentation is still there to be done
  • v7r2:
    • non-urgent issues
  • v7r3:
    • Make RSS VO-aware: PR created against v7r1, will need to be ported to v7r3
      • comments in there…
    • OAuth2 will be the main feature (see below)

WebApp:

Pilot3:

  • Updates for Jenkins (CC7 default, and CS using http) and dirac-install following the version in DIRACGrid/management

DIRACOS:

  • NTR

DIRACOS2:

  • fts-rest on py3?
    • not yet (soon?)
  • DIRACOS2 builds for ARM and Power8 (compatible with Power9?) are there. They have been also tested.
  • Question for rucio client… (issue)? (Cedric to answer)
  • Review from Andre’?
    • consider it “done”.
  • first non-pre-release today.

VMDIRAC:

  • NTR
  • Almost got it working in Certif server.

Documentation:

OAuth2:

tornado/HTTPs

  • NTR (just small PR).

management

diraccfg

  • NTR

other externals, including Rucio

  • Rucio File Catalog implementation from Belle
    • Cedric not connected today…
    • Needs (urgently?) a working implementation
    • GridPP has not yet tested it, but some comments went in the PR, especially on the point of configuration.
      • the GridPP Rucio server is multiVO, and not fully working yet.

Release planning, tests and certification

  • Certification machine

    • VMDIRAC
      • puppet updated for VMDIRAC. Please don’t update etc/dirac.cfg or bashrc by hand. puppet runs every 30 mins.
      • nearly there
    • Reminder: task for multi-vo
      •  Daniela I will write some multi-vo tests
  • Next hackathon

    • next week (18th March)
      • first hackathon for v7r3-pre1

Weekly development(s) focus

upgrading to v7r2

  • The structure with src/PROJECTNAME/code... should be adopted
  • A setup.py should be written
  • For Vanilla projects this affects COMDIRAC
    • VMDIRAC and WebAppDIRAC are only server, so no need to update immediately.
  • in LHCb we:
    • created a setup.py, and we can do pip install LHCbDIRAC
    • updated quite some code for python3

AOB and topics from Google forum

Next hackathon on March 18th: https://indico.cern.ch/event/1015938/
Next BiLD on March 25th: https://indico.cern.ch/event/1015936/

DUW10 online: https://indico.cern.ch/e/DUW10 10-11th May 2021:

  • announced “officially”, first registrants (not you!)
  • Sent a mail to presenters…

LHCbDIRAC

There are minutes attached to this event. Show them.