Go to the Indico Home Page Oldest event Older event Up to category Newer event Newest event Printable version
 
Focus on:  Hide Contributions
  • Compact style
  • Indico style
  • Indico style - inline minutes
  • Indico style - numbered
  • Indico style - numbered + minutes
  • Indico Weeks View
    • Switch to the management area of this event
    Login

    BiLD-Dev

    → Europe/Zurich
    2-R-014 (CERN)

    2-R-014

    CERN
    Federico Stagni (Conseil Europeen Recherche Nucl. (CERN)-Unknown-Unknown)
    Description
    Bi-Weekly "Loyal" DIRAC developers meeting. And, following, the LHCbDIRAC developers meeting.
    Hide

    Bi-weekly DIRAC Development meeting -- 12/12/2019

     

    At CERN: Federico, Zoltan, Christophe, Alexandre, Andre', Marko, Christopher
    On Vidyo: Ueda, Hideki, Daniela, Simon, Janusz

    Apologies: 

     

    Follow-up from previous meeting

    • no follow up for payload isolation issues and discussions, will come up again.
    • wiki created for v6r22p15 (env variable)

    DIRAC communities roundtable

    • CLIC:
      • NTR
    • LHCb:
      • NTR
    • Belle2:
      • Still running with v6r21. NTR
    • GridPP
      • NTR apart from VMDIRAC issues (see forum)

    Current situation

    DIRAC

    • v6r22:
      • NTR
    • v7r0:
      • v7r0p8 is the last one
    • v7r1:
      • NTR
    • v7r2:
      • NTR

    VM:

    • NTR

    WebApp:

    • new PR with 2 new pages have been merged and are part of v6r22p17:
      • Downtime calendar (connected to RSS)
      • Space occupancy (table for showing the SE occupancy, also taken from RSS)
    • One PR pending for CS diff

    Pilot3:

    • NTR

    DIRACOS:

    • Added lzma (merged, in master)
    • Updated version of MySQL to latest client of 5.7
      • size dropped a bit
    • Added singularity (to be merged)
      • will allow sandboxing and better isolation

    Documentation

    • fixes for extension have been merged, LHCb picked them up

    OAuth2:

    • NTR

    tornado and other externals

    • JSON encoding PRs: one "ready" (v7r1), a second built on top of it
      • several new issues added to github for fixing code in several parts
    • OpenSSL version: we are stuck with version 1.0, as 1.1 removed the possibility to specify an environment variable for RFC proxies, hopefully M2Crypto will pick up on that

    Release planning, tests and certification

    Release planning:

    • v6r22 series
      • Just on PR ready to be merged
    • v7r0 series
      • MP fixes
      • backporting of docker tests should be cherry-picked
    • v7r1 series
      • NTR
    • Integration (and v7r2)
      • MultiVO DFC PR: need last review from Zoltan

    Certification process: 

    • All PRs run GitHub Actions checks, running the full set of Integration tests
      • It's possible to run locally, just needs docker. Follow instructions at https://dirac.readthedocs.io/en/rel-v7r0/DeveloperGuide/CodeTesting/index.html#running-integration-tests-locally
      • asked question of why it fails in Actions immediately after push (on personal clone): CB will fix it.
    • puppet module ready for DIRAC certification machine
      • new machine lbcertifdirac70 (copy of lbcertifdirac7) created with it
      • the old machine can be stopped after having tested on the new one, and Trello board updated accordingly after an alias will be there

    Weekly development(s) focus

    • We should create an "integration" release+tarball automatically (nightly), and dirac-install should pick it up (e.g. for tests on integration branch)
    • management repo will be pushed from Federico in DIRACGrid
      • creates the docker images used for integration tests, upload them to GitHub package registry, but unfortunately it's not public, so will push to docker-hub too
      • creates an image for running dirac-distribution and create DIRAC tarballs
        • doc will be updated accordingly
    • Discussed how to create releases. Ideally, we want to automate as much as possible, and use github APIs to create the tag once the tests are running fine. Suggested to create a (temporary) branch where this would be done.
      • some discussions on how to create the changelog. A few things can be "copied" from DIRACOS
      • Marko will look into it, and propose a solution.

    DIRAC: current PRs and tasks being worked on, or topics from Google forum

    • v7r0:
      • PR with several fixes for FTS3
      • 2 PRs by Alexandre for WMS and computing resources should be checked by Andrei
      • #4336 will be merged if it passes the tests
    • v7r1:
      • PR created for removing the SEPrefix, some discussions followed, and Daniela will check it (might be needed by GridPP)
      • PR added an env variable for replacing DIRAC ThreadPool with native python ThreadPoolExecutor
      • JSON serialization PRs
    • v7r2:
      • # 4103 will be merged if it passes the tests
      • PRs are not passing the tests because there's no tarball...

     

    AOB

    Next DUW at KEK, 25-29 May 2020. Registration not yet open: fee is not yet finalized.

    Next BiLD in 2020!

     

    ---------------------------------------------------------------------------------------------------------------

    LHCb

    • https://docs.google.com/document/d/1U4DR-n7Omv4oKOmUaqvXkCTazsgGJG1ybfZYmat8Z9Y/edit?ts=5d8241f4
      • slowly progressing
      • We can start migrating slowly all our voboxes to CentOS7, with v6r22+DIRACOS
        • Chris: puppet templates ready
      • A new VM for testing VMDIRAC has been created
    • Deployment: we can start the new way to deploy to CVMFS also production releases, in parallel with the old one (which will keep being the default)
      • full switch from v10r0
    • LHCbDIRAC docker container: PR created, OK to support such client, will go to docker hub
    • Ganga jobs may now run in "any" container. This is not a good idea from the operational point of view. Suggested to:
      • limit the size of containers (1GB)
      • the jobs that is possible to submit this way (5)
      • print a warning
    There are minutes attached to this event. Show them.