BiLD-Dev

Europe/Zurich
2-R-014 (CERN)

2-R-014

CERN

Federico Stagni (Conseil Europeen Recherche Nucl. (CERN)-Unknown-Unknown)
Description
Bi-Weekly "Loyal" DIRAC developers meeting. And, following, the LHCbDIRAC developers meeting.

At CERN: Federico, Christophe, Andre', Marko, Louis, Gergely, Zoltan
On Vidyo: Xiaomei, Daniela, Simon, Marcelo, Yash, Hideki, Ueda

Apologies: 

 

Follow-up from previous meeting

  • in 4 weeks we'll have one more BiLD on HTTPS and DIRAC

Current situation

DIRAC

  • v6r19p24 out
    • still the main production release for many
    • just a fix for multiprocessor pilots
  • v6r20p45out
    • bug fixes
    • multi-VO transformation system
  • v6r21-pre1 out
    • in DIRAC certification
    • the next one will be called v7r0-pre1

WebApp:

  • 2 PRs for v3r1 release ready -- will be introduced with v6r20

Pilot3:

  • Andrei should include what's done for pilot2 with v6r19p24

Externals:

  • NTR

DIRACOS:

  • Marko: looks OK, just we depend on what redhat does 
  • Will be used for v7r0
    • needs some changes in dirac-install script to allow extensions of it (ongoing)
  • Using right now requirements.txt file taken from DIRAC repository
    • needs to add a file with specific fixed versions in DIRACOS repo for everything pip-installable

LCG bundles:

  • Chris found a bug in gfal2 (always there)
    • reported, a fix will eventually come (not there yet)
    • we are not sure we'll ever want to make more releases of lcgBundle anyway

REST:

  • NTR  

BOINC:

  • NTR, docs still need to be completed

PyGSI:

  • NTR

Documentation

  • Some doc added for web, more should come
    • in general, the web development should be made easier, using docker images
  • Andrei still has to migrate the rest of the documentation for the agents inside the code

DB12:

  • NTR

Release planning, tests and certification

Release planning:

  • v6r19 series
    • NTR (just a patch coming)
  • v6r20 series
    • Many PRs ready to be merged
  • v7r0 series
    • also many PRs to include
    • the list of feature request is already closed

Certification process

  • v7r0 certification: 
    • to be started, but probably seriously only in few weeks
  • https://jenkins-dirac.web.cern.ch/ is open to registrations for who needs it or wants it
    • Federico showed a way how to use jenkins for testing ongoing developments 

Weekly development(s) focus

  • Python3:
    • not yet the time for making real decisions, but awareness raised
    • we should anyway start having code that is compatible python2 and python3
    • tools are there, need to be explored
    • Federico will invite an expert to present a possible way forward
    • Should check:
      • https://github.com/DIRACGrid/DIRAC/pull/3743
        • here, only a subset of the modernize fixtures have been applied -- e.g. no use of six module
      • https://github.com/DIRACGrid/DIRAC/issues/3696
      • https://indico.cern.ch/event/694818/contributions/2985723/attachments/1680026/2702314/python-2-versus.pdf
      • https://portingguide.readthedocs.io/en/latest/index.html

DIRAC: current PRs and tasks being worked on, or topics from Google forum

  • Issue #3751: we agreed on keeping the directory operations only on the logical level
    • some development need to be done before completely removing this option (see issue)
  • Check #3710 if you want to clean the TransformationDB.DataFiles table
  • Many PRs waiting for final review

DIRAC communities roundtable

  • BES: 
    • NTR
  • CLIC:
    • Moved to v6r20 in production yesterday
  • LHCb: 
    • NTR
  • Belle2:
    • NTR

AOB

  • CHEP highlights:
    • Use of HPC growing even more
      • ATLAS converging on using Harvester
        • a kind of "small installation" that acts as a local-to-a-HPC pilot factory
        • we can probably do the same in DIRAC with what we have already, Federico thinks it's just how to configure it
    • no questions on DIRAC-Rucio interoperability!
  • Next BiLD-Dev meeting will be held in 2 weeks (August 2)

LHCbDIRAC

  • v9r2 certification
    • CC7 client needs to be tested still
    • LHCbGrid released, looks OK, but done in a patchy way
      • so we better wait for at least Ben to be back before releasing
    • release date (after consideration): July 30
      • Zoltan won't be around, Chris and Federico will
There are minutes attached to this event. Show them.
    • 10:00 10:10
      Items from Previous BiLD-Dev 10m
    • 10:10 10:25
      Current situation 15m

      General remarks

      • DIRAC 3m
      • WebApp 2m
      • Pilot 2m
      • Externals + lcgBundle 2m
      • DIRACOS 2m
      • Documentation 2m
      • pyGSI, Rest, DB12, BOINC 2m
    • 10:25 10:40
      Release planning, tests and certification 15m
    • 10:40 11:00
      Weekly development(s) focus 20m

      python3?

    • 11:00 11:10
      DIRAC: current PRs and tasks being worked on 10m

      Ongoing PRs
      - v6r20 PR
      Ongoing tasks
      - ?
      Topics from the google forum
      - ?

    • 11:10 11:20
      DIRAC Communities roundtable 10m
    • 11:20 11:30
      AOB
      Convener: Federico Stagni (CERN)
    • 11:30 11:45
      LHCbDIRAC 15m