Go to the Indico Home Page Oldest event Older event Up to category Newer event Newest event Printable version
 
Focus on:  Hide Contributions
  • Compact style
  • Indico style
  • Indico style - inline minutes
  • Indico style - numbered
  • Indico style - numbered + minutes
  • Indico Weeks View
    • Switch to the management area of this event
    Login

    BiLD-Dev

    → Europe/Zurich
    Federico Stagni (CERN)
    Description

    Bi-Weekly "Loyal" DIRAC developers meeting. And, following, the LHCbDIRAC developers meeting.

    Join Zoom Meeting
    https://cern.zoom.us/j/91083694183?pwd=ZkdDY1I1YkJVc2o3UTdBY1BRZE15UT09

    Meeting ID: 910 8369 4183
    Passcode: 12345678
    One tap mobile
    +41432107108,,91083694183# Switzerland
    +41315280988,,91083694183# Switzerland

    Dial by your location
            +41 43 210 71 08 Switzerland
            +41 31 528 09 88 Switzerland
            +41 43 210 70 42 Switzerland
            +33 1 8699 5831 France
            +33 1 7037 2246 France
            +33 1 7037 9729 France
    Meeting ID: 910 8369 4183
    Find your local number: https://cern.zoom.us/u/artUfAdNB

    Join by SIP
    91083694183@188.184.89.188
    91083694183@188.185.118.153

    Join by H.323
    188.184.89.188
    188.185.118.153
    Meeting ID: 910 8369 4183
    Passcode: 12345678

    working notes
    Hide
     
     

    BiLD (Bi-weekly DIRAC Development meeting) – 17/03/2022

    At CERN (in our offices) Federico, Alexandre, André, Christopher
    On Zoom: Daniela, Janusz, Ruben, Simon, Hideki, Michel Hernandez Villanueva
    Apologies: Andrii, Andrei

    Follow-up from previous meetings

    DIRAC communities roundtable

    GridPP:

    Daniela+Simon+Janusz

    • NTR

    LHCb:

    Federico+Christophe+Christopher+Alexandre+Ruben

    • Moved to Antares (CTA)

    ILC/CLIC/FCC:

    André

    • RAL Storage moving to Voldemort (that which shall not be named), thinking about staging via SMS or FTS

    Belle2

    Hideki

    • Moved to v7r2 (python2 still)
    • Deploying ES
      • Q: do you plan to keep ES7 and OpenSearch support?
        • largely does not depend from DIRAC. In 8.0 we have moved to using the OpenSearch clients
      • Q: version of OpenSearch
        • right now tests use 1.0.0
    • Question on MQ usage: how many servers/setup
      • At CERN we use the existing ActiveMQ setup (managed by IT, via K8)

    Topics from GitHub/Discussions or Google forum

    DIRAC releases

    • v7r2p42:
      • RequestManagementSystem
        • FIX: (#5917) filter by SourceSE in ReplicateAndRegister preparation
        • FIX: (#5917) Change RMS.Operation sourceSEList and targetSEList default to empty list
      • WorkloadManagementSystem
        • FIX: (#5905) Matcher: fixed parsing of (required) tags
    • v7.3.19:
      • WorkloadManagementSystem
        • FIX: (#5910) using a DErrno for no match found
    • 8.0.0a17:
      • WorkloadManagementSystem
        • CHANGE: (#5933) removed last SubmitPool entries
        • NEW: (#5925) added TornadoWMSAdministratorHandler
      • RequestManagementSystem
        • FIX: (#5926) removing python2 support
      • FrameworkSystem
        • CHANGE: (#5931) DBs stop using BLOB types in favor of TEXT types
        • NEW: (#5901) move ProxyManager to HTTPs
      • DataManagementSystem
        • NEW: (#5927) TornadoDataIntegrityHandler

    DIRAC projects

    DIRAC:

    Issues by milestone:

    • v7r2:
      • NTR
    • v7r3:
      • NTR
    • v8.0:
      • Moved a few issues to v8.1
    • v8.1:
      • 8 open issues already here

    Other issues:

    PRs discussed:

    WebApp:

    • All PRs reviewed and merged

    Pilot:

    • Janusz Revived the code for the remote pilot logger to store log messages in a remote SE. Trying to log as early as possible.
      • The logger sends single (JSON) message to an HTTP service. On the server side an agent is in charge of a first post-processing of the messages. Log files are closed and uploaded to a Storage Element (being S3 or others… does not fully matter)
      • 2 PRs, one for Pilot (devel branch) and one for DIRAC (https://github.com/DIRACGrid/DIRAC/pull/5961)
        • Pilot: reused some of the Pilot code that was written in the past
        • DIRAC: the service, it was objected, should not just fill up a local file, as this would not work when the service is distributed. Instead, the service should forward to an ElasticSearch backend the JSON messages as ES documents
        • what is coded with the local file could be used for a first patchy test (to be set up) but not in a production environment (and the PR should not be merged as is)

    DIRACOS2:

    • Restored Micromamba, released DIRACOS 2.21
      • much faster
      • Linux ARM and Linux PPC were broken, Apple MacOS added.

    Documentation:

    OAuth2:

    tornado/HTTPs

    • from previous meeting Andrii We can eventually have special classes in Configuration and BundleDelivery to pass binary data - to be discussed.

    management

    • new issue added (from previous meetings discussions)

    diraccfg

    • NTR

    COMDIRAC

    • NTR

    DB12

    Alexandre

    • DB12 now correctly corrects the scores
    • Multi-core allocations: created a PR that should solve the issues.
    • Next: analyze Job Power/DIRAC Power ratio?

    Rucio

    • Existing PR ready to be merged (not used by anyone ATM)

    Release planning, tests and certification

    • 8.0 release

      • Currently missing:
        • a few critical issues, but for OAuthZ we don’t want to improve ATM
        • BLOB->TEXT everywhere
        • Replace gMonitor object
      • Not in a hurry to have the release out. Target: 3/4 more weeks.

    • Certification machines

      • lbcertifdirac70 machine:
        • NTR
      • lbcertifdiracoauth machine:
        • NTR

    • Next hackathon(s)

      • Suggestion is that for the next hackathon to use again lbcertifdirac70

    AOB

    Next hackathon on March 24th
    Next BiLD on March 31st

    Mail from the Lawyer of CNRS (update for the consortium agreement). Montpellier was not in, some more fixes. Will go through another round.

    DIRAC Workshop. 9-10 May 2022. Will stay virtual, and the presentations will stay in the mornings (similar schedule to last years’). The question is if we want to ALSO meet at CERN, and add non-ZOOM sessions in the afternoons.

    LHCbDIRAC

    • from previous meeting Christopher the BookkeepingDB assumes a certain detector (Run2). This is wrong because it is not anymore the same detector, but is it useful at all?
      • Christophe I made a MR changing the way the condition path is constructed. While doing this I found out that there is quite some data that we can’t access
        • Christopher it might be that some changes that I have started doing locally are fixing the described issue
      • Christophe we should really make the description unique
    • v10r3: deploy board in https://trello.com/b/kzUKdMts/deploy-v10r3
    • LHCbDIRAC hackathon based on 8.0
      • can start looking into it, target first week of April
     
    There are minutes attached to this event. Show them.
      • 10:00 → 10:10
        Items from Previous BiLD-Dev 10m
      • 10:10 → 10:20
        DIRAC Communities roundtable 10m
      • 10:20 → 10:30
        DIRAC releases 10m
      • 10:30 → 10:55
        DIRAC projects 25m
        • DIRAC
        • WebApp
        • Pilot
        • DIRACOS
        • DIRACOS2
        • VMDIRAC
        • Documentation
        • OAuth2
        • tornado/HTTPs
        • other externals (include Rucio)
        DIRAC certification
      • 10:55 → 11:00
        Release planning, tests and certification 5m
        DIRAC PRs
        google forum
      • 11:00 → 11:15
        Weekly development(s) focus 15m
      • 11:15 → 11:25
        AOB
        Convener: Federico Stagni (CERN)
      • 11:25 → 11:40
        LHCbDIRAC 15m