BiLD-Dev

→ Europe/Zurich
2-R-014 (CERN)

2-R-014

CERN

Federico Stagni (Conseil Europeen Recherche Nucl. (CERN)-Unknown-Unknown)
Description
Bi-Weekly "Loyal" DIRAC developers meeting. And, following, the LHCbDIRAC developers meeting.

At CERN: Federico, Andre', Marko, Alexandre, Florent, Chris, Vladimir, Philippe, Joel
On Vidyo: Andrei, Simon Fayer, Ueda, Milosz, Luisa

Apologies: Wojtek

 

Follow-up from previous meeting

  • NTR

Current situation

DIRAC:

  • v6r17p29 is still the last production version (no new versions since last week)
  • v6r18 wiki and release notes have been merged in v6r19
    • v6r20 wiki started
  • v6r19-pre8 is out
    • Still problems with RSS
      • Decided to re-implement fully ResourceManagementDB
      • ResourceStatusDB should also be updated, but not for v6r19
    • Using externals v6r6p3 and lcgBundle v12r5
    • Today we'll make v6r19-pre9

WebApp:

  • NTR

Pilot:

  • Created devel branch with fixes and new tests, merged already in master branch after verification
    • Looks stable, used in production VMs in LHCb
  • We can introduce Pilot3 as optional from the SiteDirectors from next version

Externals:

  • v6r6p3 still the latest (for v6r19) for both SLC6 and CC7

LCG bundles:

  • No updates
    • [from previous meeting] New Version v12r5 compiled, in testing with DIRAC v6r19 pre versions
      • We discourage its use for now
      • SLC6 version looks OK (inserted in DIRAC certification, both client and server)
      • CC7 version is KO (all dms commands fail because at least gfal2 - but not only - are not fully compiled, at least not statically compiled)
        • Not in our hands, more news from librarians meeting in September

PyGSI:

  • NTR

Release planning, tests and certification

Release planning:

  • v6r17 series
    • p20 will be created today
  • v6r19 series
    • v6r19-pre9 will be created including PRs already reviewed OK
    • For tags and MultiProcessor support: PRs from Andrew can be closed, PR from Xiaomei to be updated

Certification process: 

  • One more certification hackhathon will be done tomorrow morning
    • We use Trello boards and Slack channels
    • Trello boards are copied from https://trello.com/b/cp8ULOhQ/dirac-certification-template in, e.g. https://trello.com/b/Q7QKMjbI/dirac-v6r19-pre7
    • Each trello card is a task to do for the certification process, many of them contain explanation of the steps to be done
    • For each update on every trello card, the slack channel https://lhcbdirac.slack.com/messages/C3AGWCA8J/ gets an update
  • CI tests:
    • NTR

Weekly development(s) focus

  • Using JSON as marshaling protocol [Florent]
    • json module extended for dealing with tuple, long and datetime
    • still performances look better than DEncode
    • can be used for DISET serialization only, no need to remove DEncode
    • need 3 releases for keeping backward compatibility (see presentation attached)
      • minimal code changes
      • make PR with proposed changes
    • agreed that more tests would be needed
  • gLogger -> logging updates [Alexandre] 
    • re-stated the changes done (see presentation attached)
    • Tested and verified in certification also the part on logs centralization
    • Should be certified also the part on centralizing messages in ElasticSearch
    • Asked about performance decreases when using multiple backends
      • what was studied was the effect of multiple backends compared to one.
      • What is not studied is the effect of running in DEBUG compared to running in INFO
    • Followed discussion about when to use verbose instead of debug mode.

DIRAC: current PRs and tasks being worked on

  • Nothing else to report (all PRs discussed)

AOB

  • Next BiLD-Dev meeting will be held in 3 weeks (September 14) and agenda is already there

 

LHCbDIRAC

  • pipilines now include pylint, unit tests, and coverage
    • done on each developers' repositories 
There are minutes attached to this event. Show them.
    • 10:00 → 10:10
      Items from Previous BiLD-Dev 10m
      • v6r19 is the target
      • pyGSI under DIRACGrid + new version
    • 10:10 → 10:20
      Current situation 10m
      • Externals v6r6p3
      • lcgBundle v12r5
        problems with gfal and CC7
      • DIRAC v6r19-pre6
        certification hackhathons
    • 10:20 → 10:35
      Release planning, tests and certification 15m
      DIRAC certification
    • 10:35 → 10:55
      Weekly development(s) focus 20m
      • possible changes of serialization protocol
      • logging update
    • 10:55 → 11:05
      DIRAC: current PRs and tasks being worked on 10m

      Ongoing PRs
      - v6r19 PR
      Ongoing tasks
      -
      Topics from the google forum
      - ?

    • 11:05 → 11:15
      AOB
      Convener: Federico Stagni (CERN)
    • 11:15 → 11:30
      LHCbDIRAC 15m