BiLD-Dev

Europe/Zurich
Federico Stagni (Conseil Europeen Recherche Nucl. (CERN)-Unknown-Unknown)
Description

Bi-Weekly "Loyal" DIRAC developers meeting. And, following, the LHCbDIRAC developers meeting.

Join Zoom Meeting
https://cern.zoom.us/j/91857549918?pwd=S2c1ZmFpYVNDOGk4YjMrZmhOeGROUT09

Meeting ID: 918 5754 9918
One tap mobile
+41315280988,,91857549918# Switzerland
+41432107042,,91857549918# Switzerland

Dial by your location
        +41 31 528 09 88 Switzerland
        +41 43 210 70 42 Switzerland
        +41 43 210 71 08 Switzerland
        +33 1 7037 2246 France
        +33 1 7037 9729 France
        +33 1 8699 5831 France
Meeting ID: 918 5754 9918
Find your local number: https://cern.zoom.us/u/aUn2ex94J

Join by SIP
91857549918@188.184.110.70
91857549918@188.184.89.188

Join by H.323
188.184.110.70
188.184.89.188
Meeting ID: 918 5754 9918
 

BiLD (Bi-weekly DIRAC Development meeting) – 13/08/2020

At CERN: Nobody, of course!
On Zoom: Federico, Alexandre, Andrii, Aymane, Christophe, Christopher, Daniela, Simon, Vladimir

Apologies: André, Marko

Follow-up from previous meeting

After OAuth pres:

  • REST interface: RFC for discussion? (see today)
    • Not yet started it [Andrei]
  • Basic OAuth library:
    • Chris Burr started drafting something. "More or less I have a proposal, hope to open an issue before holidays
  • RFC for semantic of components?
    • not started yet [Andrei/Federico/Andrii]

DIRAC communities roundtable

GridPP:

  • NTR

CLIC:

  • Nobody there to report

LHCb:

  • Ran 2 more certification hackathons using v7r1. Looks OK, just not much time now for production (which will be in September)

France Grilles:

  • NTR

EGI:

  • NTR

Belle2:

  • Nobody there to report

Nica:

  • Nobody there to report

Juno:

  • Nobody there to report

Current situation

DIRAC

  • v6r22:
    • v6r22p33
      • M2Crypto/pyGSI related fixes
  • v7r0:
    • v7r0p32
      • JobAgent - define DIRAC_JOB_PROCESSORS environment to number of processors allocated for the user job
      • WebAppDIRAC v4r0p30
  • v7r1:
    • v7r1p10
      • NTR
  • v7r2:
    • NTR

WebApp:

  • Do update to latest version!

Pilot3:

  • Merged one PR for better monitoring of scripts
  • One draft PR open for tests of various types of CEs

DIRACOS:

VM:

  • NTR

Documentation:

  • NTR

OAuth2:

  • Discussed last week.
  • Documention is in progress. Only done for now for REST.

tornado and other externals

  • tornado: today’s topic
  • M2Crypto: …end of line for the fixes?
    • still one open PR on M2Crypto side

management

diraccfg

  • NTR

Release planning, tests and certification

  • Big PRs for v7r2 are not yet reviewed. Will need to wait for them before really starting certifications. Given the holiday period, we are talking about September.

Weekly development(s) focus

  • HTTPs (presentation on the agenda)
  • Previous BiLD/presentations/discussions/links also on the agenda

Summary:

  • The advantages are clear (use of Standard library, interrogate services without installing DIRAC).
  • Services would need to be migrated by inheriting from the new TornadoService.
  • Clients don’t need to be migrated.
  • CS protocol flag needs to change to HTTPs
    Dev:
  • One route per service, then POST verb with remote method name and parameters
  • One tornado process runs all the services
    • Exception for master CS
      Issues:
  • Input/output must be JSON-compatible (some tasks still open)
  • FileTranfer not fully implemeted
    • Tornado is not for that. Need to use something else
  • Threads are not allowed
    • Need to fit into ASync style. Callbacks “poor man” solution.
  • Global variables can’t stay (gLogger, gConfig)
  • DIRACOS should include:
    • patched Tornado 5.1 (fork will come in GitHub/DIRACGrid)
    • we need to rely on M2Crypto, for now, and there’s also a patch for it

Discussion:

  • Tornado is maybe not what we want. But it’s right now the way to have https protocol. Maybe it won’t stay in the future, maybe it could be replaced one day, but in the meantime DIRAC can start speaking HTTPS

Conclusion:

  • Merging this PR should be OK. Running https services need to be manual.
  • OAuth PR: what’s done here seems to fit there too.

DIRAC: current PRs and tasks being worked on, or topics from Google forum

PRs:

  • v7r0:
    • Few PRs ready to be merged, non-problematic
  • v7r1:
    • One large-ish PR with simple fixes for scripts
  • v7r2:
    • NTR

On issues:

  • NTR

Topics from Google forum:

  • Little activity lately

AOB

CERN policies still forbid travel. Can’t know yet when there will be next DUW.
EGI workshop in November will go fully virtual. 1 day DIRAC session? probably - AT in contact.

Next BiLD in 5 weeks.


LHCbDIRAC

  • Oracle instantclient to be updated in production too.
There are minutes attached to this event. Show them.
    • 10:00 AM 10:10 AM
      Items from Previous BiLD-Dev 10m
    • 10:10 AM 10:20 AM
      DIRAC Communities roundtable 10m
    • 10:20 AM 10:35 AM
      Current situation 15m

      General remarks

      • DIRAC 3m
      • WebApp 2m
      • Pilot 2m
      • DIRACOS 2m
      • VMDIRAC 2m
      • Documentation 2m
      • OAuth2 2m
      • tornado, M2Crypto, other externals 2m
    • 10:35 AM 10:50 AM
      Release planning, tests and certification 15m
    • 10:50 AM 11:10 AM
      Weekly development(s) focus 20m

      HTTPs services in DIRAC

      Speaker: Christophe Haen (CERN)
    • 11:10 AM 11:20 AM
      DIRAC: current PRs and tasks being worked on 10m

      Ongoing PRs
      - v6r22 PRs
      - v7r0 PRs
      - v7r1 PRs
      - v7r2 PRs
      Ongoing tasks
      - ?
      Topics from the google forum
      - ?

    • 11:20 AM 11:30 AM
      AOB
      Convener: Federico Stagni (CERN)
    • 11:30 AM 11:45 AM
      LHCbDIRAC 15m