Focus on:  Hide Contributions
Login

BiLD-Dev

→ Europe/Zurich
Federico Stagni (Conseil Europeen Recherche Nucl. (CERN)-Unknown-Unknown)
Description

Bi-Weekly "Loyal" DIRAC developers meeting. And, following, the LHCbDIRAC developers meeting.

Join Zoom Meeting
https://cern.zoom.us/j/91857549918?pwd=S2c1ZmFpYVNDOGk4YjMrZmhOeGROUT09

Meeting ID: 918 5754 9918
One tap mobile
+41315280988,,91857549918# Switzerland
+41432107042,,91857549918# Switzerland

Dial by your location
        +41 31 528 09 88 Switzerland
        +41 43 210 70 42 Switzerland
        +41 43 210 71 08 Switzerland
        +33 1 7037 2246 France
        +33 1 7037 9729 France
        +33 1 8699 5831 France
Meeting ID: 918 5754 9918
Find your local number: https://cern.zoom.us/u/aUn2ex94J

Join by SIP
91857549918@188.184.110.70
91857549918@188.184.89.188

Join by H.323
188.184.110.70
188.184.89.188
Meeting ID: 918 5754 9918
 

working notes
Hide

BiLD (Bi-weekly DIRAC Development meeting) – 30/07/2020

At CERN: Nobody, of course!
On Zoom: Federico, Andrii, Andrei, André, Alexandre, Aymane, Cedric, Christopher, Daniela, Hideki, Janusz, Marko, Xiaomei, Raja, Ruslan (Belle2), Simon

Apologies:

Follow-up from previous meeting

  • DIRAC releases on CVMFS:
    • Andrei setup a nightly cron job which is updating the CVMFS with all those that are in releases.cfg
    • Also removes from CVMFS those that are removed (still a lot of clean-up to do)
    • Also updates the global cfg file, the tarballs, and JSON for pilot
    • Keeping it in local for now, but should be put in GitHub Actions, and then use secrets
      • Marko: I will prepare you a container in management repo
  • TLS issue: https://github.com/DIRACGrid/DIRAC/issues/4675 has been closed (should be solved in modern versions)
  • Error in Pilot related to locale (https://github.com/DIRACGrid/Pilot/issues/112). Deployed: can it be confirmed that fixes the original issue by Nica?

DIRAC communities roundtable

GridPP:

CLIC:

  • NTR

LHCb:

  • Running certification using v7r1. Looks OK, just not much time now for production.

France Grilles:

  • CC/Lyon started providing pilot of ES service. One peculiarity is that it needs logging in with certificate. Need DIRAC patch for using it.

EGI:

  • Setting up token-based AuthN server side. Still discussions going on.

Belle2:

  • Migration to v6r22: not yet, because of local maintenance.
  • Faced one problem: attempted to use latest lcgBundle (r11)
    • Problem in RMS when using Davix protocol - backported the version
      • AT: why RMS? - HM: thread never answering (gfal2 library issue?)
    • DIRACOS should be of course the solution, but not certified for davix.

Nica:

  • NTR

Juno:

  • NTR

Current situation

DIRAC

  • v6r22:
    • v6r22p32
      • ElasticSearchDB: backported update() function from v7r0.
      • ARCComputingElement - reserve the number of cores as defined in the NumberOfProcessors configuration parameter
  • v7r0:
    • v7r0p29
      • M2Crypto fixes (not final)
      • Configuration: add --cfg option for config files, deprecating old non-option version.
      • Few fixes for TS and PS
      • Hack in Slurm to stop jobs running out of time on multi-processors resources
  • v7r1:
    • v7r1p7
      • dirac-install is getting the globalDefaults.cfg from CVMFS if available, loads the installation configuration from etc/dirac.cfg if it is present.
      • NEW command dirac-install-extension - new command to add an extension to an existing installation
      • WMS: DownloadInputData no longer fails if getFileMetadata fails for the first SE, tries others if available
  • v7r2:
    • NTR

WebApp:

Pilot3:

DIRACOS:

  • xroot5?
  • Last version includes the newest version of M2Crypto
  • RN: why the less command doesn’t work? - MP: it’s a known bug. It’s a bit too much work to fix it, vi works. AT: just don’t source bashrc… not many solutions.

VM:

Documentation:

  • NTR

OAuth2:

tornado and other externals

  • M2Crypto: a PR for DIRAC is in WIP

management

  • see above (beginning)

diraccfg

  • NTR

Release planning, tests and certification

  • Big PRs for v7r2 are not yet reviewed. Will need to wait for them before really starting certifications.

Weekly development(s) focus

The discussion was on 3 points:

  • how to implement REST in DIRAC. 2 options, right now:
    1. use “Chris PR”
    2. use current WebApp framework
      AT: the 2 implementation are just different. We need to sort out how we want to go in the future. Will open an RFC for discussion.
  • base library used for implementing the OAuth protocol. 2 options:
    1. use an existing one (e.g. https://github.com/oauthlib/oauthlib or https://github.com/lepture/authlib)
    2. create our one – this is what’s done right now
      Chris B will share his experience with Andrii about using the existing external ones
  • semantic of components
    FS: why VOMS is not part of Resource? - AT: we don’t have ATM a way to define its semantics, maybe we need yet another Resource type. - FS: we will leave for long time in a mixed environment, we need clear definitions. - AT: I will create an RFC for discussion.

DIRAC: current PRs and tasks being worked on, or topics from Google forum

PRs (there was no time for further discussions):

  • v7r0:
    *
  • v7r1:
    *
  • v7r2:
    *

On issues:

Topics from Google forum:

  • ElasticSearch setup:
  • How to reset files in “processed” status: question from Xiaomei – …?
  • long jobs - proxy renewal: question for ARC and FG setup
  • Stage out retry and how do I test this: generated one issue https://github.com/DIRACGrid/DIRAC/issues/4694 (see above)
  • API::job submission:: specify the number of replicas (and SEs) for output: basically a feature request, do we tranform it into a GitHub issue?

AOB

CERN policies still forbid travel. Can’t know yet when there will be next DUW.
EGI workshop in November will go fully virtual. 1 day DIRAC session? probably - AT in contact.

Next BiLD in 2 weeks. There will be a 5 weeks with no BiLD after that.

LHCbDIRAC

  • The meeting was too long and the LHCBDIRAC part was not discussed

  • LHCbWebAppDIRAC: new Analysis Productions page (merged?)

There are minutes attached to this event. Show them.
Powered by Indico v3.3.7-pre