BiLD-Dev

→ Europe/Zurich
2-R-014 (CERN)

2-R-014

CERN

Federico Stagni (Conseil Europeen Recherche Nucl. (CERN)-Unknown-Unknown)
Description
Bi-Weekly "Loyal" DIRAC developers meeting. And, following, the LHCbDIRAC developers meeting.

At CERN: Federico, Christophe, Zoltan, Philippe, Joel, Wojtek, Andre', Marko, Hamza, Xiaomei
On Vidyo: Marcelo, Raja, Milosz, Daniela, Simon

Apologies: Andrei

 

Follow-up from previous meeting

  • FTS3 policy: fine
  • autopep8: need to find the proper strategy

Current situation

DIRAC:

  • v6r19p7 is the last production version
    • v6r19p6 was buggy, and not properly announced/removed, apologies

WebApp:

  • The new release procedure of WebApp has been merged, will be part of v6r20 

Pilot:

  • NTR

Externals:

  • NTR

LCG bundles:

  • v13r0 includes python packages like pyparsing and futures, which should be anyway in the externals. To verify
  • still not moved its "development" to GitHub. This would also answer the recurrent question of the content of each release

REST:

  • Wojtek will create PRs as soon as possible for:
    • choosing among different authentication methods (OAuth and X509)
    • adding the pilots logging interface
  • The connection with CS-based authZ will be done later
  • Tornado proposes some unit/integration tests which look easy to use, and that can be integrated in Travis or Jenkins. Maybe also for WebApp.  

BOINC:

  • NTR

PyGSI:

  • NTR

Documentation

  • Improved creation of scripts documenttion

DB12:

  • NTR

Release planning, tests and certification

Release planning:

  • v6r19 series
    • v6r19p8 should be out today
    • deprecated decorator added, can be used already
  • v6r20 series
    • pre9 will be out today
    • the changes in REST, once done and merged, will be included in a pre-release, which we can then install on the DIRAC certification machine
    • planned release in January 2018

Certification process: 

  • v6r20 certification: 
    • Using trello and slack as usual, not much progress these last 3 weeks
  • The lhcb-jenkins server has been retired
    • jenkins-dirac server is OK

Weekly development(s) focus

  • RAL is moving from Castor to ECHO. No SRM.
    • DIRAC is not fully ready yet. Mostly operational but with some developments to be done

DIRAC: current PRs and tasks being worked on

  • #3554 (Transactional bulk jobs submission). Comment is that jobs that can't be submitted should not be considered as "Failed"
  • pyGSI -> M2Crypto: maybe for v6r21
    • Made several tests
    • Removed the legacy proxies part
    • Added a flag to use it or not, based on an environment variable
    • sockets part being added
  • dirac-wms vs dirac-admin scripts: time to change the names?
    • Philippe will make a plan for changing names and positions altogether

DIRAC communities roundtable

  • ILC: NTR
  • LHCb: Moved to v6r19
    • some issues of slowness in jobs submission due to RSS, fixed now
  • BES3: still on v6r18, they will move ASAP
    • Using some MP resources but not at production level
  • GridPP: v6r19 latest patch
    • Using MP resources successful (some issues setting them up), with Pilot2
    • Trying to use GPU resources, still some work to do

AOB

  • Next BiLD-Dev meeting will be held in 2 weeks (December 14)

LHCbDIRAC

  • "prod" simlink of LHCbGrid moved to v13r0
    • also updated the documentation
  • new jenkins-dirac is used, old jenkins retired
    • Need to make the slaves not visible from outside
    • job logs should be closed 
    • and puppetize the cernVMs
  • LHCbDIRAC v9r1 can include only changes to ProductionRequest and be out next week
  • LHCbDIRAC v9r2 will include changes for platform selection and require full certification (out next yer)
There are minutes attached to this event. Show them.
    • 10:00 → 10:10
      Items from Previous BiLD-Dev 10m
    • 10:10 → 10:25
      Current situation 15m

      General remarks

      • DIRAC 3m
      • WebApp 2m
      • Pilot 2m
      • Externals 2m
      • lcgBundle 2m
      • Documentation 2m
      • pyGSI, Rest, DB12, BOINC 2m
    • 10:25 → 10:40
      Release planning, tests and certification 15m
    • 10:40 → 11:00
      Weekly development(s) focus 20m

      RAL "ECHO" (AKA no SRM)

    • 11:00 → 11:10
      DIRAC: current PRs and tasks being worked on 10m

      Ongoing PRs
      - v6r19 PR
      Ongoing tasks
      - ?
      Topics from the google forum
      - ?

    • 11:10 → 11:20
      DIRAC Communities roundtable 10m
    • 11:20 → 11:30
      AOB
      Convener: Federico Stagni (CERN)
    • 11:30 → 11:45
      LHCbDIRAC 15m