BiLD-Dev

Europe/Zurich
Federico Stagni (Conseil Europeen Recherche Nucl. (CERN)-Unknown-Unknown)
Description

Bi-Weekly "Loyal" DIRAC developers meeting. And, following, the LHCbDIRAC developers meeting.

Join Zoom Meeting
https://cern.zoom.us/j/91857549918?pwd=S2c1ZmFpYVNDOGk4YjMrZmhOeGROUT09

Meeting ID: 918 5754 9918
One tap mobile
+41315280988,,91857549918# Switzerland
+41432107042,,91857549918# Switzerland

Dial by your location
        +41 31 528 09 88 Switzerland
        +41 43 210 70 42 Switzerland
        +41 43 210 71 08 Switzerland
        +33 1 7037 2246 France
        +33 1 7037 9729 France
        +33 1 8699 5831 France
Meeting ID: 918 5754 9918
Find your local number: https://cern.zoom.us/u/aUn2ex94J

Join by SIP
91857549918@188.184.110.70
91857549918@188.184.89.188

Join by H.323
188.184.110.70
188.184.89.188
Meeting ID: 918 5754 9918
 

BiLD (Bi-weekly DIRAC Development meeting) – 22/10/2020

At CERN: A few of us, but each in our office ;-)
On Zoom: Federico, André, Andrii, Anil Panta (Belle2), Cedric, Christophe, Christopher, Daniela, Hideki, Marko, Simon, Vladimir, Xiaomei

Apologies: Alexandre, Andrei

Follow-up from previous meeting

management devs

 Marko+AndréWe are discussing moving the DIRAC Monitoring of Components that is used in ILCDIRAC to vanilla DIRAC. Issues with REA.

DIRAC v7r1 hackathons

DIRAC communities roundtable

GridPP:

  • NTR

CLIC:

  • on v6r22 + DIRACOS
  • enabled “IPv6 ready” for some servers to let the CERN DNS announce the machines with IPv6(?) (disabled by default). All working fine.
    • Couldn’t use one IPv6 only CE
  • Start looking at multi-core usage (mostly >2GB memory requirements)
    • Just for ARC CE for the moment
    • no PoolCE usage yet
  • Starting integrating FCC VO and usage.
    • Maybe use that for a reason to take over the DIRAC/Workflow/Module developments
    •  Federicodeveloped long time ago by me, the Script and ModuleBase should be fine. The other 2 modules were never fully tested. There should be no real changes in the last few versions.

LHCb:

  • Still on v7r0. Hackathons for v7r1 tomorrow, then hopefully production.

France Grilles:

  • NTR

EGI:

  • NTR

Belle2:

  • Adopting Rucio: https://github.com/DIRACGrid/DIRAC/issues/4495
  •  Cedricwe’ll try with DIRACOS, which we are using with v6r22 in integration server

  •  CedricI will make a PR soon, and then I would like to present it at the BiLD meeting

JINR:

  • Just 1 VO using JINR service for now, running MC. 500K jobs, 120 TB generated.
  • NTR on the technical side.

Juno, BES3:

  • Mostly questions about redundancy

Current situation

DIRAC

  • v6r22p35:
    • exclude fuse from df (watchdog checks)
  • v7r0p36:
    • NTR
  • v7r1p14:
    • BDII2CSAgent: several changes from PR #4752
    • Added support for ES7 #4747
    • remove backward compatibility for BaseSE defined in StorageElements section #4755
  • v7r2-pre16:
    • Included tornado/HTTPs PR

WebApp:

  • New PR for PilotMonitor
    • failures in this PR should be fixed separately
  • Use ngnix for data transfer: working on it
    • logging to be verified (merge)

Pilot3:

  • Changed defaults for PoolCE #118
  • Added test for various CE types (Jenkins, “submitAndMatch” test)

DIRACOS:

  • xroot5? no news

DIRACOS2:

VMDIRAC:

  • NTR

Documentation:

  • Alexandre pushed tutorial update for installing WMS (v7r0)

OAuth2:

  • The PR #4650 needs to be rebased and then target v7r3
    • Will use HTTPS/tornado as base

tornado/HTTPs

 FedericoWhich DIRACOS version should be used?
 ChristopheFirst, let’s try a hackathon with the DIRACOS version specific for tornado (already done), to make sure that it doesn’t break anything. Then we can maybe make a new specific version

other externals

  • NTR

management

reminder Marko prepared a container to be used for CVMFS (auto)deploy: https://github.com/DIRACGrid/management/packages/342716 (also in https://hub.docker.com/repository/docker/diracgrid/dirac-cvmfs). For using it from GitHub Actions template in: https://github.com/DIRACGrid/management/pull/19 [Andrei + Federico]

reminder DIRACOS should be expanded on CVMFS [Andrei]

diraccfg

  • NTR

Release planning, tests and certification

 Daniela

  • Added the gridpp VO
  • some issues with RSS, but RSS is not multi-VO
    waiting for Janusz
  • TS should be fine as multi-VO
  • let’s just run all the tests and we take from there
  • multi-VO metadata: does it work? specific tests should be added
  •  FedericoMonitoring (on ES) is probably also not multi-VO aware
  • Next hackathon October 29th ?
    • First to verify that the multi-VO part works
      • or at least it doesn’t break anything
    • v7r2: no python3 client yet, we will use the DIRACOS for tornado
    • Let’s not change the board for now, it will be enough

reminder Jenkins: diracci6-02 “slave” machine will be removed
* this is used by several tests: https://jenkins-dirac.web.cern.ch/label/dirac_ci_6/
* Should move to use CC7 node https://jenkins-dirac.web.cern.ch/computer/diracci7-02/

  • v7r2:
    1. Included HTTPs
    2. Include Python 3 PRs (only the most recent one, as the others are included)
    3. functional client on python 3 [in dev]
    4. Option of DIRACOS2 [in dev]

Weekly development(s) focus

DIRAC: current PRs and tasks being worked on, or topics from Google forum

PRs No other PRs to discuss

Issues

Topics from Google forum

  • NTR (forum quiet lately)

AOB

EGI webinar tomorrow by Andrei.

Next BiLD in 2 weeks.


LHCbDIRAC

  • ES: before moving to ES7 we need to reduce the number or indices.

 Federicodidn’t have time to look into it this week, will try next week.

  • Hackathon tomorrow, based on a patch of v7r1 that Andrei should create this afternoon
There are minutes attached to this event. Show them.
    • 10:00 AM 10:10 AM
      Items from Previous BiLD-Dev 10m
    • 10:10 AM 10:20 AM
      DIRAC Communities roundtable 10m
    • 10:20 AM 10:35 AM
      Current situation 15m
      • DIRAC
      • WebApp
      • Pilot
      • DIRACOS
      • DIRACOS2
      • VMDIRAC
      • Documentation
      • OAuth2
      • tornado/HTTPs
      • other externals
    • 10:35 AM 10:50 AM
      Release planning, tests and certification 15m
    • 10:50 AM 11:10 AM
      Weekly development(s) focus 20m
    • 11:10 AM 11:20 AM
      DIRAC: current PRs and tasks being worked on 10m

      Ongoing PRs
      - v6r22 PRs
      - v7r0 PRs
      - v7r1 PRs
      - v7r2 PRs
      Ongoing tasks
      - ?
      Topics from the google forum
      - ?

    • 11:20 AM 11:30 AM
      AOB
      Convener: Federico Stagni (CERN)
    • 11:30 AM 11:45 AM
      LHCbDIRAC 15m