BiLD-Dev

→ Europe/Zurich
2-R-014 (CERN)

2-R-014

CERN

Federico Stagni (Conseil Europeen Recherche Nucl. (CERN)-Unknown-Unknown)
Description
Bi-Weekly "Loyal" DIRAC developers meeting. And, following, the LHCbDIRAC developers meeting.

Bi-weekly DIRAC Development meeting -- 10/10/2019

 

At CERN: Federico, Zoltan, Christophe, Alexandre, Andre', Christopher, Vladimir
On Vidyo: Andrei, Igor, Daniela, Simon, Janusz, Hideki, Ueda

Apologies: Marko, Sasha, Xiaomei, 

 

Follow-up from previous meeting

  • vyPR presentation:
    • the general feeling is that it's not something useful as of now
    • main concerns are:
      • lack of python3 support
      • general support
      • ROI unclear
    • we might have a look at it again in the future, once we've migrated to tornado for our services

DIRAC communities roundtable

  • CLIC:
    • NTR, still on v6r21
  • LHCb:
    • NTR
  • GridPP:
    • Testing v7r0
      • issues in separating the DIRAC environment from that of the users' applications
        • originated 1 PR that want to split the environment
    • Using VMDIRAC for running on Openstack clouds, and SingularityCE
  • Belle2:
    • issues reported on SiteDirector #4262 an #4263 fixed
    • still on v6r21, but started moving to v622
    • started looking into logs aggregation
  • FranceGrilles:
    • Moved to v7r0, but still sending Pilot2
    • Rather smooth, the only thing found out is that RESTDIRAC is broken now (tornado version differ)
  • EGI
    • v6r22 used for most of the services
    • once PRs for v7r1 will be merged, EGI services will start moving to it

Current situation

DIRAC

  • v6r22:
    • only minor bug fixes
  • v7r0:
    • a tag for v7r0p2 created.
    • LHCb did a short hackathon with a release on top of DIRAC v7r0. Quite smooth.
  • v7r1:
    • second hackathon last week
      • rather smooth

VM:

  • Should use Pilot3 instead of Pilot2, and use a new version together with DIRAC v7r0

WebApp:

Pilot3:

  • always creating a local pilot.cfg file for pilot configuration options

DIRACOS:

  • v1r4 created and used in DIRAC v7r0 releases

Documentation

  • NTR

OAuth2:

  • NTR

tornado and other externals

  • PR #4261 has been rebased on v7r1 and it's almost ready for merge (needs some more test)

Release planning, tests and certification

Release planning:

  • v6r22 series
    • NTR
  • v7r0 series
    • NTR
  • v7r1 series
    • Hackathon done last week. Maybe another one next week (than there won't be any during CHEP period)
  • Integration (and v7r2)
    • NTR

Certification process: 

  • NTR

Weekly development(s) focus

 

  • Information system: BDII, GoCDB, CRIC...?
    • VOs have the BDII2CSAgent installed and find it a necessary tool
  • BDII is reported as dying
    • LHCb is the only LHC VO that still use it 
    • CRIC advertised as its replacement, but seems to be for LHC VOs only
  • CRIC2CSAgent might be added, but we better first check if there's the need of it
    • Vladimir will check what's the source of info of CRIC
    • If the source is GOCDB itself then we might do the work in DIRAC by simply extending GOCDB2CSAgent 

 

  • SystemLogging:
    • VOs don't seem particularly interested in keeping it, as the ELK stack does more and better than that.
    • Federico will circulate info on how to stop it, and create a PR to integration branch to remove it.

DIRAC: current PRs and tasks being worked on, or topics from Google forum

  • https://github.com/DIRACGrid/DIRAC/pull/4286 is for running integration tests in CI
    • can be merged!

AOB

Next BiLD in 2 weeks from now

 

---------------------------------------------------------------------------------------------------------------

LHCb

  • https://docs.google.com/document/d/1U4DR-n7Omv4oKOmUaqvXkCTazsgGJG1ybfZYmat8Z9Y/edit?ts=5d8241f4
    • document with a collection of operations, mostly for DB
    • MySQL:
      • several things to do (see doc), the first can be using the strict mode everywhere, also on MySQL 5.6
      • The migration to 5.7 will need a downtime, to be planned
      • Chris will talk with Ignacio to better define the plan
    • Oracle will also need to be updated, maybe a downtime is needed
      • Zoltan to verify and add it to the doc
    • In the meantime: migrate slowly all our voboxes to CentOS7, with v6r22+DIRACOS
      • Chris: started re-shuffling the puppet templates
  • LHCbDIRAC v10r0:
    • no news and no need for hackathon, but we'll need a release soon in LHCb-Certification setup for running MP jobs at CINECA HPC 
There are minutes attached to this event. Show them.
    • 10:30 → 10:40
      Items from Previous BiLD-Dev 10m
    • 10:40 → 10:50
      DIRAC Communities roundtable 10m
    • 10:50 → 11:05
      Current situation 15m

      General remarks

      • DIRAC 3m
      • WebApp 2m
      • Pilot 2m
      • DIRACOS 2m
      • VMDIRAC 2m
      • Documentation 2m
      • OAuth2 2m
      • tornado, M2Crypto, other externals 2m
    • 11:05 → 11:20
      Release planning, tests and certification 15m
    • 11:20 → 11:40
      Weekly development(s) focus 20m

      Information system:
      BDII, GoCDB, CRIC...?

      • VOs: do you have the BDII2CSAgent installed?
      • BDII is dying
      • CRIC2CS to add
      • CEs info
        queue length, benchmark, ...?
      • extend GOCDB2CS ?

      SystemLogging:

      • VOs: do you use it?
      • the ELK stack does more and better than that.
    • 11:40 → 11:50
      DIRAC: current PRs and tasks being worked on 10m

      Ongoing PRs
      - v6r22 PRs
      - v7r0 PRs
      - v7r1 PRs
      - v7r2 PRs
      Ongoing tasks
      - ?
      Topics from the google forum
      - ?

    • 11:50 → 12:00
      AOB
      Convener: Federico Stagni (CERN)
    • 12:00 → 12:15
      LHCbDIRAC 15m