BiLD-Dev

→ Europe/Zurich
2-R-014 (CERN)

2-R-014

CERN

Federico Stagni (Conseil Europeen Recherche Nucl. (CERN)-Unknown-Unknown)
Description
Bi-Weekly "Loyal" DIRAC developers meeting. And, following, the LHCbDIRAC developers meeting.

At CERN: Federico, Christophe, Philippe, Zoltan, Vladimir, Hamza, Marko, Vladimir, Andre, Wojtek
On Vidyo: Simon, Marcelo, Hideki

Apologies: Andrei, Luisa, Ueda

 

Follow-up from previous meeting

  • lcgBundle: nothing new for the moment, but we said we actually need a new version
    • which should come max on Monday
  • Release procedure for webApp updated

Current situation

DIRAC

  • v6r19p16 is the last production version
    • just fixes for the pilot, and matching on MP resources
      • tested by GridPP, looks working OK for their need now

WebApp:

  • The new release procedure of WebApp part of v6r20 is OK
  • Test for moving WebApp to ExtJS6 ongoing, finished
    • Will be for 4.0 version of WebAppDIRAC
    • NOT for DIRAC v6r20, later
    • Extensions would need to be adapted

Pilot:

  • Wojtek created a PR for pilot logging
    • still want to add more complete integration tests for both REST and MQ cases, to be used together RESTDirac and our MQ components, so keeping the [WIP] flag for this moment.
  • Jenkins tests are now easier to use and can be used for testing the above
  • Pilot3 will become optional for v6r20 SiteDirectors, using a flag [Federico]
    • Currently Pilot3 files are uploaded to the web portal
      • there is a security breach here that needs to be fixed. [Zoltan]
    • Needs to update the documentation for that

Externals:

  • v6r6p5 created. Used in v6r20, looks OK for now
    • only for SLC6 for now, so that certification can keep going
  • For current Externals,  Federico created https://github.com/DIRACGrid/Externals/pull/37 which compile python with pip
    • then old ez_setup and other deprecated stuff has been removed
  • What Chris and Marko are doing for the lcgBundle can one day completely replace the Externals too

LCG bundles:

  • current production version is v13r0, but 4 or 5 new versions of packages of the Grid middleware are available, so new version of the lcgBundle will come. Also needing new xroot and to re-include LFC support.
    • This was not done but it's now high priority, and needs to come NOW
  • Chris and Marko are look into re-compiling everything, starting from the source RPMs provided for RedHat 6. Then, for DIRAC to use it we need to change dirac-install to accept a new flag, e.g. DIRACOS flag that can be used for pointing to this new bundle[Zoltan to help]
    • for the moment only creating one version (no client/server)
    • need to use Jenkins or other CI system

REST:

  • Wojtek will create PRs as soon as possible for:
    • choosing among different authentication methods (OAuth and X509)
    • adding the pilots logging interface
    • Delayed because of issues discovered in the DIRAC stomp interface
    • ...and because of code refurbishment
  • The connection with CS-based authZ will be done later
  • Tornado proposes some unit/integration tests which look easy to use, and that can be integrated in Travis or Jenkins. Maybe also for WebApp.  

BOINC:

  • NTR, docs still need to be completed

PyGSI:

  • NTR

Documentation

  • NTR

DB12:

  • NTR

Release planning, tests and certification

Release planning:

  • v6r19 series
    • make v6r19p17 before next week
  • v6r20 series
    • pre15 out
    • contains all the new features, so now it's only about testing
    • the flag for enabling bulk submission transaction can be removed [Federico] 
    • FTS -> FTS3: we should develop a script to move files from old to new system [Hamza with the help of Chris]
    • we enabled shortly the flag that tells us what JSON can't parse, and got lots of errors
      • we need a first visual evaluation and remove the obvious ones [Chris]

Certification process: 

  • v6r20 certification: 
    • Using trello and slack as usual
    • Some updates as usual
    • We need to (re)start all tests with the new lcgBundle 

Weekly development(s) focus

  • Glue2 support not in Bdii2CSAgent neither [Andre + Vladimir]
    • https://github.com/DIRACGrid/DIRAC/issues/3633
      • HTCondorCEs at CERN now not advertised anymore in Glue1 
    • Glue2 seems hard to understand
  • ARC CEs
    • still sites need to install patches for DIRAC to understand its content
    • glue2 support still not there, not sure if with that it would work [Vladimir to look into it]
    • [Raja + Andrei + Daniela] to look in details

DIRAC: current PRs and tasks being worked on

  • NTR

DIRAC communities roundtable

  • GridPP:
    • NTR
  • BES: 
    • NTR
  • ILC:
    • NTR, just Hamza leaving in one month
  • LHCb: 
    • MySQL 5.7 comes with new plans, some problem for certain queries being investigated by DBoD
  • INFN:
    • NTR
  • DIRAC4EGI:
    • NTR
  • Belle2:
    • NTR.

AOB

  • New website to be for diracgrid.org: find it at fstagni.web.cern.ch
  • Next BiLD-Dev meeting will be held in 2 weeks (March 22)

LHCbDIRAC

  • We should right now concentrate on testing v6r20
    • Philippe's MR merged
  • at most 4 students this summer: proceeding
    • summer student project accepted, selection started now

 

There are minutes attached to this event. Show them.
    • 10:00 → 10:10
      Items from Previous BiLD-Dev 10m
    • 10:10 → 10:25
      Current situation 15m

      General remarks

      • DIRAC 3m
      • WebApp 2m
      • Pilot 2m
      • Externals 2m
      • lcgBundle 2m
      • Documentation 2m
      • pyGSI, Rest, DB12, BOINC 2m
    • 10:25 → 10:40
      Release planning, tests and certification 15m
    • 10:40 → 11:00
      Weekly development(s) focus 20m
    • 11:00 → 11:10
      DIRAC: current PRs and tasks being worked on 10m

      Ongoing PRs
      - v6r19 PR
      Ongoing tasks
      - ?
      Topics from the google forum
      - ?

    • 11:10 → 11:20
      DIRAC Communities roundtable 10m
    • 11:20 → 11:30
      AOB
      Convener: Federico Stagni (CERN)
    • 11:30 → 11:45
      LHCbDIRAC 15m