Go to the Indico Home Page Oldest event Older event Up to category Newer event Newest event Printable version
 
Focus on:  Hide Contributions
  • Compact style
  • Indico style
  • Indico style - inline minutes
  • Indico style - numbered
  • Indico style - numbered + minutes
  • Indico Weeks View
    • Switch to the management area of this event
    Login

    BiLD-Dev

    → Europe/Zurich
    Federico Stagni (CERN)
    Description

    Bi-Weekly "Loyal" DIRAC developers meeting. And, following, the LHCbDIRAC developers meeting.

    Join Zoom Meeting
    https://cern.zoom.us/j/91083694183?pwd=ZkdDY1I1YkJVc2o3UTdBY1BRZE15UT09

    Meeting ID: 910 8369 4183
    Passcode: 12345678
    One tap mobile
    +41432107108,,91083694183# Switzerland
    +41315280988,,91083694183# Switzerland

    Dial by your location
            +41 43 210 71 08 Switzerland
            +41 31 528 09 88 Switzerland
            +41 43 210 70 42 Switzerland
            +33 1 8699 5831 France
            +33 1 7037 2246 France
            +33 1 7037 9729 France
    Meeting ID: 910 8369 4183
    Find your local number: https://cern.zoom.us/u/artUfAdNB

    Join by SIP
    91083694183@188.184.89.188
    91083694183@188.185.118.153

    Join by H.323
    188.184.89.188
    188.185.118.153
    Meeting ID: 910 8369 4183
    Passcode: 12345678

    working notes
    Hide
     

    BiLD (Bi-weekly DIRAC Development meeting) – 16/09/2021

    At CERN: Federico, Alexandre, Christopher, Christophe, André, Vladimir (in our offices…)
    On Zoom: Andrei, Andrii, Cedric, Daniela, Hideki, Simon, Yuji, Janusz
    Apologies:

    Follow-up from previous meetings

    • hackathon 8th September, 7.4.0a1:
      • quick py3 hackathon on lbcertifdirac70 box
      • no big issues encountered

    DIRAC communities roundtable

    GridPP:

    Daniela

    • Running fine, no upgrades
    • 3 issues:
      • proxy expiring warnings (will make an issue)
      • PR 5392 : permission control for accounting (prob PR is OK)
      • FC kept hanging (at about 20 requests/seconds) --> SSL ERRORs
        • Christopher this is more or less the same threshold that we have seen in LHCb, suggestion is to try to apply #5394
        • another suggestion is to simply run more services

    ILC/CLIC/FCC:

    André

    • NTR

    LHCb:

    Federico+Christophe+Christopher+Alexandre

    • Running python3 pilots everywhere (only waiting for VAC sites to update)
      • Federico Q: what about VMDIRAC?
        • from 7.3 the startup scripts should be changed (do not bother with VMDIRAC used by v7r2).
    • Fighting the whole week against overloading of master CS. Initially we thought that moving to py3 pilots was the issue, but we could not relate it. We ended up:
    • Ran a first LHCbDIRAC hackathon using v7r3p1 as base (py2 server, py3 client and pilots), no issues encountered.
      • will soon move this installation to py3 server

    EGI:

    Andrei

    • Moved most of the services to v7r2, one service has to be left to v7r1 (for robot proxy upload).
      • a fix is ready, will be made soon.
    • Setting up v7r3p1 server with the goal to run http services
      • Need to set up WMS and ProxyManager with http for the OpenMOLE community (scala language) – this needs to be developed.

    Belle2:

    Hideki, Cedric, Ueda

    • from previous meeting Try to use BySize plugin - only input by integer GB which is too much. Possible solution is to use floats for the size
    • Observed some finished jobs that are again “Running” - asked to post on forum.

    BES

    Xiaomei

    • from previous meeting Asking for the transfers monitoring, this does not exist yet
      • in the pipeline but not for now - requirements and a plan of work would be needed
      • Chris FTS has a grafana dashboard, if you are using it
    • globalDefaults issue from forum: will be fixed

    JINR

    Igor

    • NTR

    DIRAC releases

    • v7r2p25:

      • purely fixes

    • 7.3.1:

      • Created the final 7.3 (patch 7.3.1): nothing new wrt the previous pre version, mostly fixes for tornado based services
      • NEW: (#5000) Support for Python 3 server installations
      • NEW: VMDIRAC separate project is merged into the core DIRAC project
      • NEW: (#5042) RSS: Multi-VO mode of operation support
      • NEW: (#4903) Added WMS (Jobs) State Machine
      • NEW: (#5214) Added HTTPs JobManager, JobMonitor, JobStateUpdate services
      • NEW: (#5314) Add a PilotStatus module to clearly define the pilot status in the code
      • CHANGE: (#4715) removing env variable DIRAC_USE_NEWTHREADPOOL
      • CHANGE: (#4937) removed dirac-agent, dirac-service, dirac-executor scripts (use ‘_’ counterparts instead)
      • CHANGE: (#5337) removed src/DIRAC/Core/scripts/dirac-install.py
      • CHANGE: (#5110) Dropped Glue2Only and Glue2URLS options from Bdii2CSAgent, Glue2 is now the only way
      • CHANGE: (#4303) Removed completely the SystemLogging
      • CHANGE: StatesMonitoring agent removed
      • CHANGE: (#5313) Docs: replace the theme by the sphinx_rtd_theme

    • 7.4.0a1

      • Dedicated to Tokens and integration on new IdPs
      • Python3 servers installations “defaults”

    • Q: should we re-brand 7.4 to 8.0 (and just skip 7.4)?

      • Andrei if we are fully ready with dropping python2, then why not
      • after brief discussion: we agree on moving directly to version 8.0, skipping 7.4, and dropping with it python2 support: the consensus is that maintaining python2 and DIRACOS would be more painful
      • all 7.4 issues will be moved to 8.0

    DIRAC projects

    DIRAC:

    Issues by milestone:

    • v7r1:
      • We don’t want to put more effort in this version, so urged to finish off with the existing issues (mostly documentation)
    • v7r2:
      • NTR
    • v7r3:
      • NTR
    • v7r4:
      • will be merged in 8.0
    • v8.0:
      • NTR

    Other issues:
    *

    PRs discussed:

    WebApp:

    • from previous meetings Problem with empty ComponentMonitoring data - to be checked
      • need to use the DIRAC service, not go directly to ElasticSearch
    • Started to make PRs to use BaseRequesthandler, for WebApp handlers
      • Christophe more documentation is needed
    • Next release: 5.0 (python3 only)

    Pilot:

    DIRACOS:

    DIRACOS2:

    • from previous meetingfts-rest on py3?
      • fts3-rest is added
      • the “official” client will be worked “soon”
    • Still missing rucio
      • 2 issues on their github still open

    Documentation:

    OAuth2:

    tornado/HTTPs

    • NTR

    management

    • a few minor PRs to be merged

    diraccfg

    • NTR

    COMDIRAC

    • NTR, but Pierre Gay should be ping-ed

    DB12

    Alexandre

    • Port to python3:
      • In python3: long == int and this generates slightly different scores.
      • We ran DB12 on many different processors: dividing the score by 1.16 seems to correct the issue for Intel processors but not for AMD ones.
      • To be decided exactly how to move forward (by next BiLD)

    other externals, including Rucio

    • Change in the rucio file catalog client itself, requires a config file – this discussion is not specific for DIRAC, though.
    • PR: if Cedric approves it, we merge it

    Release planning, tests and certification

    • Certification machines

      • lbcertifdirac70 machine:
        • no changes
      • lbcertifdiracoauth machine:
        • no changes

    • Next hackathon(s)

      • on lbcertifdirac70

    AOB and topics from Google forum

    Next hackathon: on September 23rd:
    Next BiLD on September 30th: https://indico.cern.ch/event/1077642/

    LHCbDIRAC

     
    There are minutes attached to this event. Show them.