Go to the Indico Home Page Oldest event Older event Up to category Newer event Newest event Printable version
 
Focus on:  Hide Contributions
  • Compact style
  • Indico style
  • Indico style - inline minutes
  • Indico style - numbered
  • Indico style - numbered + minutes
  • Indico Weeks View
    • Switch to the management area of this event
    Login

    BiLD-Dev

    → Europe/Zurich
    Federico Stagni (Conseil Europeen Recherche Nucl. (CERN)-Unknown-Unknown)
    Description

    Bi-Weekly "Loyal" DIRAC developers meeting. And, following, the LHCbDIRAC developers meeting.

    Join Zoom Meeting
    https://cern.zoom.us/j/91083694183?pwd=ZkdDY1I1YkJVc2o3UTdBY1BRZE15UT09

    Meeting ID: 910 8369 4183
    Passcode: 12345678
    One tap mobile
    +41432107108,,91083694183# Switzerland
    +41315280988,,91083694183# Switzerland

    Dial by your location
            +41 43 210 71 08 Switzerland
            +41 31 528 09 88 Switzerland
            +41 43 210 70 42 Switzerland
            +33 1 8699 5831 France
            +33 1 7037 2246 France
            +33 1 7037 9729 France
    Meeting ID: 910 8369 4183
    Find your local number: https://cern.zoom.us/u/artUfAdNB

    Join by SIP
    91083694183@188.184.89.188
    91083694183@188.185.118.153

    Join by H.323
    188.184.89.188
    188.185.118.153
    Meeting ID: 910 8369 4183
    Passcode: 12345678

    working notes
    Hide

    BiLD (Bi-weekly DIRAC Development meeting) – 27/05/2021

    At CERN: Still, nobody :/
    On Zoom: Federico, Alexandre, André, Andrei, Andrii, Christophe, Christopher, Daniela, Hideki, Igor, Janusz, Marko, Simon, Yuji
    Apologies: Christopher

    Follow-up from previous meetings

    • Ran hackathon last week on 7.3.0a8:
      • trello board here
      • ComponentMonitoring still looks problematic.
        •  Christophe needs to be further developed to be really used
          • we should screen it out (remove it from the WebApp and maybe remove it from docs)
      • VMDIRAC not tested ( Daniela would like to note that any member of the dteam VO can run the VMDIRAC certification test)
    • PRs merging strategy?
      •  Marko done…?
        • PR request created: https://github.com/DIRACGrid/DIRAC/pull/5139
          • Discussion on who’s in charge of making the sweeper-created Pull Request. Everyone thinks it should be the developer of the original PR.
        •  Federico we should tried it out in https://github.com/DIRACGrid-test/DIRAC first
        • discussion about when to start applying it.
          • conflicting ideas if using black or not.
            • main points against (from Andrei and André):
              • the advantage is not very clear
              • in a while we might need to update the style again
          • make a poll on using black or not?

    DIRAC communities roundtable

    GridPP:

     Daniela

    • Running v7r1p39 in production.
    • User issues: "Creating new processes randomly fails with EPoll: Bad file descriptor polling for events, opened in May 2020 by Chris B: https://github.com/xrootd/xrootd/issues/1198 This seems to be becoming more prevalent, could we revisit this issue ?
      •  Christophe there’s an env variable to get around - it should be removed by the bashrc created in the pilot by dirac-install
        • suggestion is to get DIRAC from CVMFS
    • User issues: We managed to get a dodgy file entry in the DFC (file only present in one of two tables) and Simon would like a cleaner way to remove these entries and/or a clearer error message than hacking the database and this: 
      > dirac-dms-remove-files /gridpp/user/d/daniela.bauer/api/specialpath/testfile.1592236837.LCG.RAL-LCG2.uk.txt
Failed to remove replica. /gridpp/user/d/daniela.bauer/api/specialpath/testfile.1592236837.LCG.RAL-LCG2.uk.txt 
{'FileCatalog': "Server error while serving removeReplica: 'FileID'"} Failed to remove 1 files with error:  {'FileCatalog': "Server error while serving removeReplica: 'FileID'"}
      Simon would like to discuss this briefly in this meeting.
    • Administrator issues: Does DIRAC still issue warning for short lifetime certificates (we had this problem originally with short autogererated certificates from DUNE) and if yes, how is this going to work for token?

    CLIC:

     André

    • Finally made the PR with “ComponentSupervisionAgent” f.k.a. “MonitoringAgent”

    LHCb:

     Federico+Chris+Chris+Marko

    • Using v7r1 with DIRACOS v1r21 for a while now, and we have seen no problems
    • from last LHCbDIRAC hackathon, it seems that we are ready to move to v7r2 in production (will be done next week)

    EGI (+ France Grilles):

     Andrei

    • from previous meeting M2Crypto and VOMS extension follow-up (issue)
      *
    • from previous meeting problem with retrieving the output for ARC
      • DIRACOS v1r21 OK?
    •  Federico I see (from DUW) ESCAPE is part of the EGI users. At CHEP they showed a “data lake” with no mention of DIRAC…?

    Belle2

     Hideki

    • Ready to move on to v7r0, prob next week.

    JINR

     Igor

    • TS was successfully configured. MC + analysis, looks good
    • idea of providing DIRAC for wider range of users. App for sending jobs?
      • recompiled WebApp, using docker

    DIRAC releases

    • v7r1p40:
      • CHANGE: (#5123) Read X509_CERT_DIR from central place
      • Some HTCondorCE fixes
    • v7r2p7:
      • FIX: (#5133) The RPC Client class should now be threadsafe
    • v7r3-pre8 (7.3.0a8):
      • NEW: Added WMS State Machine

    DIRAC projects

    DIRAC:

    Issues by milestone:

    PRs discussed:

    WebApp:

    • NTR

    Pilot3:

    • NTR

    DIRACOS:

    • xroot5?
      • no updates
    • condor 9.0?

      • 9.0 presentation

        • File transfer (input, output): new plugins
        • https, Amazon S3…
        • Add information about GPU usage
        • More security (ID Tokens…)
        • Tool for monitoring jobs in real time: condor_watch_q
          • by the way, the HTCondorCE calls the pilotDB to get information about status of the pilots in a specific Site.
            Is there any particular reason? I wonder if it wouldn’t be better to get the status of the jobs/pilots directly from HTCondor, seems to dependent to the pilots.

      • 9.1 presentation

        • New command names: condor <noun> <verb>

          • condor job status, condor job submit

        • Legacy tools (condor_q, condor_submit, …) not going anywhere…

        • Might need to take into account these new commands in the future relases?

        • Briefly talked about HTCondor interacting with HPC with no external connectivity (CMS on Mare Nostrum)

        • New things probably not useful for DIRAC (at least for the moment):

          • Possibility to create set of jobs but do not see any use-case with DIRAC
          • Better integration of GPU
          • Kubernetes support
          • Docker universe becomes Container universe: more general

      •  Alexandre Overall, HTCondor 9.0 brings many new features but most of them are probably not really useful for DIRAC.

    DIRACOS2:

    • from previous meetingfts-rest on py3?
      • NTR
    • from previous meeting Question for rucio client… (issue)?
      • NTR
    • updating to python3.9

    VMDIRAC:

    • NTR

    Documentation:

    • some issues with Sphinx 4, should be fixed by now.

    OAuth2:

    tornado/HTTPs

    • NTR

    management

    • from previous meeting 1 task left for deploying on CVMFS
      • Not yet done  Marko I will come back to it
    • from previous meeting Add same check to VOMS_DIR and VOMSES as there is for CERT
      • This PR was merged
      • Asked to add the created bashrc to qa branch of puppet modules
        • also done, not propagated to master

    COMDIRAC

    • NTR

    other externals, including Rucio

    • NTR

    Release planning, tests and certification

    •  Federico approved a true multiVO test for certification: https://github.com/DIRACGrid/DIRAC/pull/5157 (–> trello card)

    • Certification machine(s)

      • authcertif machine:
        • https://github.com/DIRACGrid/DIRACOS/issues/178 (py3 only ?)
        • We install DIRAC py3 on authcertif
        • from previous meeting we should probably need admin access to dteam IaM – probably also managed by Andrea
          •  Andrii I am in VO and I have an account

    • Next hackathon

      • next week (3rd June) – regular
      • “oath+py3” hackathon in next 2 weeks - tbd exactly when

    Weekly development(s) focus

    • DUW 2 weeks ago
      • I have not received any comment
        • any comment from you, right now?
        •  Federico I am personally rather satisfied
      • Some notes taken here: https://demo.hedgedoc.org/Z7ET6a8VRPeVaVJhKJ2qTA
      • Some interest in Rucio, but the integration is not fully complete.
      • AuthZ is also a topic that received interest.
      • Most wanted features:
        • Pilot logs is very wanted feature by many communities and installations
          • GridPP also proposing implementation
        • Component supervision Agent ported from ILC
      • Reminder: don’t hesitate to use GitHub for issues!
        •  Federico I have not created any issue, if you have some, please do.
      • Recordings added to the Indico page

    AOB and topics from Google forum

    Next hackathon on June 3d
    Next BiLD on June 10th

    LHCbDIRAC

    • No news
    • python3 client on CVMFS
      • LHCbPilot would need to add a flag
    There are minutes attached to this event. Show them.