Go to the Indico Home Page Oldest event Older event Up to category Newer event Newest event Printable version
 
Focus on:  Hide Contributions
  • Compact style
  • Indico style
  • Indico style - inline minutes
  • Indico style - numbered
  • Indico style - numbered + minutes
  • Indico Weeks View
    • Switch to the management area of this event
    Login

    BiLD-Dev

    → Europe/Zurich
    Federico Stagni (Conseil Europeen Recherche Nucl. (CERN)-Unknown-Unknown)
    Description

    Bi-Weekly "Loyal" DIRAC developers meeting. And, following, the LHCbDIRAC developers meeting.

    Join Zoom Meeting
    https://cern.zoom.us/j/91083694183?pwd=ZkdDY1I1YkJVc2o3UTdBY1BRZE15UT09

    Meeting ID: 910 8369 4183
    Passcode: 12345678
    One tap mobile
    +41432107108,,91083694183# Switzerland
    +41315280988,,91083694183# Switzerland

    Dial by your location
            +41 43 210 71 08 Switzerland
            +41 31 528 09 88 Switzerland
            +41 43 210 70 42 Switzerland
            +33 1 8699 5831 France
            +33 1 7037 2246 France
            +33 1 7037 9729 France
    Meeting ID: 910 8369 4183
    Find your local number: https://cern.zoom.us/u/artUfAdNB

    Join by SIP
    91083694183@188.184.89.188
    91083694183@188.185.118.153

    Join by H.323
    188.184.89.188
    188.185.118.153
    Meeting ID: 910 8369 4183
    Passcode: 12345678

    working notes
    Hide

    BiLD (Bi-weekly DIRAC Development meeting) – 25/03/2021

    At CERN: Still, nobody :/
    On Zoom: Federico, André, Andrei, Andrii, Christophe, Christopher, Daniela, Hideki, Igor, Janusz, Simon, Xiaomei
    Apologies: André (will be ~15 minutes late), Alexandre

    Follow-up from previous meeting

    DIRAC communities roundtable

    GridPP:

     Daniela

    • NTR

    CLIC:

     André

    • Updated to v7r1p34 (directly from v6r22)
      • pilot3=false for now
      • Forgot to adapt to the changes for InputDataQuery -> MetaDataQuery in the TransformationManagerHandler (for v7r0), e.g. createTransformationInputDataQuery removed, no deprecation warning, not enough testing on my side :(
      • (TPC Castor to EOS still doesn’t work with root, gsiftp works, castor will disappear next month anyway) Whatever happened to xrootd 5?

    LHCb:

     Federico+Chris+Chris+Marko

    • CTA in production at CERN
    • We have put Singularity as InnerCE at 1 CE per Tier1.
      • didn’t find specific issues

    EGI + France Grilles:

     Andrei

    • merged EGI and FG together
      • all preliminary work before Monday (FG hosting)
        • including moving to v7r1 (was v7r0)
      • merged the CFG for users and resources
        • dropped many CREAM CEs in the meanwhile, replaced with ARC and HTCondor CEs
      • then changed the DNS alias
        • and jobs started to flow
      • Issues encountered:
        • robot certificates
          • Some delegations were non-RFC -> Fixed
          • One had a max of delegations to 2 -> path length error
            • problematic for OpenSSL recent versions
          • Another cert needs more investigation, debugging
            • hint: remove the .pyo files at every iteration
            • due to this, running with M2Crypto disabled
        • Many ARC CEs are configured to not deliver the pilot proxy to the current directory
          • as a result, jobs can’t download ISB. Contacted the sites directly.
          •  André: adding XRSLExtraString = (runtimeenvironment="ENV/PROXY") to CE configs also sorts out the issue
      • only ~1000 running jobs for now, few communities have not yet restarted
      • Accounting restarted
      • Migrated the DFC, rather smoothly
    • SiteDirectors with Pilot3: not yet switched on for everybody

    Belle2

     Hideki

    BES3/Juno:

     Xiaomei

    • running transformations:
      • how to use FailoverRequest module?
        • Federico will answer by mail
      • how to use FailoverSEs?
        • Chris wants to backport a function from LHCbDIRAC (which is also in ILCDIRAC).

    JINR:

     Igor

    • No changes in DIRAC infrastructure
    • Can we monitor the resources usage of user jobs?
      •  Federico In DIRAC the JobWrapper starts a Watchdog that is based on psutil (from v7r1) for internal monitoring. There’s a potential of enhanced it for reporting info to an external DB (e.g. into ES or InfluxDB).
      •  Christopher suggestion to look into https://github.com/HSF/prmon
    • in NICA: tried with sending info to InfluxDB with separate script
      • some performance issues, so limited.
    • JINR switched all CREAM CEs to ARC

    DIRAC releases

    • v7r1p33 + v7r1p34:
      • NEW: (#4960) add --FTSOnly and --ExcludeSE options to dirac-dms-protocol-matrix
    • v7r2:
      • Created first release
        • Using WebAppDIRAC:v4r2-pre6, will be using v4r2 final from v7r2p1
    • v7r3-pre1:
      • first pre-release
      • still using OAuthDIRAC:v0r1-pre6, time to remove it

    DIRAC projects

    DIRAC:

    Issues by milestone:

    • v7r1:
      • almost only documentation is still there to be done
    • v7r2:
      • a few issues still there, not urgent
    • v7r3:
      • all issues are still to be addressed

    PRs discussed:

    Discussed PR #5035 on DIRAC scripts re-writing. The suggestion is to split this PR in 2 parts, one with the logic changes and a second with the bulk changes.

    WebApp:

    • OAuth2 PR created
    • One PR to be rebase to v4r2 (with hackathon fixes)
    • Christophe opened ticket for v4r0 and accounting WebApp

    Pilot3:

    DIRACOS:

    • xroot5? (last issue there)
      • to be done, still. It’s in DIRACOS2.
    • also should check for adding arc6 (right now still on arc5 client, this poses a few issues with some sites)

    DIRACOS2:

    • from previous meetingfts-rest on py3?
      • No news
    • from previous meeting Question for rucio client… (issue)?
      • NTR
    • first non-pre-release (2.0) is there since a few weeks

    VMDIRAC:

    • NTR

    Documentation:

    • Using a conda environment when building the docs (PR merged)
    • Building docs on RTD for PRs
      • using the same thing used for the publication
        • so, you can browse the docs inside a PR

    OAuth2:

    tornado/HTTPs

    • refactored the DFC handler into 1 (part of PR)

    management

    • from previous meeting only one task left for deploying on CVMFS

      •  Marko should come from DIRAC main repo.

    •  Federico we need to start taking care of the deployments of DIRACOS2 and of pip-installable DIRAC clients

    COMDIRAC

    • Adapting to py3 setup

    other externals, including Rucio

    • from previous meetings Rucio File Catalog implementation from Belle
      • Needs (urgently?) a working implementation
    • PR Reviewed a few times, not all the “editorial” comments have been replied
      • Cedric busy with other business
    • It situation persists, then it’s maybe better to branch off
    •  Janusz: I am in the process of interfacing to our own Rucio server
      • started from the changing the client a bit

    Release planning, tests and certification

    • Certification machine

      • VMDIRAC
        • NTR
      • Reminder: task for multi-vo

    • Next hackathon

      • next week (1st April)
        • v7r3-pre2?

    Weekly development(s) focus

    NTR

    AOB and topics from Google forum

    Next hackathon on April 1st: https://indico.cern.ch/event/1022567/
    Next BiLD on April 8th: https://indico.cern.ch/event/1022566/

    LHCbDIRAC

    There are minutes attached to this event. Show them.