Go to the Indico Home Page Oldest event Older event Up to category Newer event Newest event Printable version
 
Focus on:  Hide Contributions
  • Compact style
  • Indico style
  • Indico style - inline minutes
  • Indico style - numbered
  • Indico style - numbered + minutes
  • Indico Weeks View
    • Switch to the management area of this event
    Login

    BiLD-Dev

    → Europe/Zurich
    Federico Stagni (CERN)
    Description

    Bi-Weekly "Loyal" DIRAC developers meeting. And, following, the LHCbDIRAC developers meeting.

    Join Zoom Meeting
    https://cern.zoom.us/j/91083694183?pwd=ZkdDY1I1YkJVc2o3UTdBY1BRZE15UT09

    Meeting ID: 910 8369 4183
    Passcode: 12345678
    One tap mobile
    +41432107108,,91083694183# Switzerland
    +41315280988,,91083694183# Switzerland

    Dial by your location
            +41 43 210 71 08 Switzerland
            +41 31 528 09 88 Switzerland
            +41 43 210 70 42 Switzerland
            +33 1 8699 5831 France
            +33 1 7037 2246 France
            +33 1 7037 9729 France
    Meeting ID: 910 8369 4183
    Find your local number: https://cern.zoom.us/u/artUfAdNB

    Join by SIP
    91083694183@188.184.89.188
    91083694183@188.185.118.153

    Join by H.323
    188.184.89.188
    188.185.118.153
    Meeting ID: 910 8369 4183
    Passcode: 12345678

    working notes
    Hide

    BiLD (Bi-weekly DIRAC Development meeting) – 10/06/2021

    At CERN: Still, nobody :/
    On Zoom: Federico, André, Andrei, Andrii, Cedric, Christopher, Daniela, Hideki, Janusz, Marko, Simon, Yuji, Vladimir
    Apologies:

    Follow-up from previous meetings

    DIRAC communities roundtable

    GridPP:

     Daniela

    • Updates in https://github.com/xrootd/xrootd/issues/1198 (EPoll: Bad file descriptor polling for events)
      • solution on sight (will need a new release of xrootd)
        •  Federico+Christopher DIRACOS has anyway not updated to version 5 so probably this will be much later for us
      • env variable removal looks like working fine in any case
    • User request: Could “listCatalog” that is currently commented out in the DIRAC API please be enabled ? (Why do I get the feeling that the answer will be: Sure, can you do it ?)
      •  Andrei implemented in the console. Create an issue, assign it on me.
    • We haven’t had any more input from the user about the xrootd 3rd party copy problem, I assume they are all on holiday.

    CLIC:

     André

    • Started looking at v7r2, migration end of July
      • Trying first to get the python2 tests to work, will look at the python3 client later

    LHCb:

     Federico+Chris+Chris+Marko

    • Deployed v7r2 in production
      • main issues was an incompatibility with v7r1 fixed in PR
      • following the deploy, other issues were investigated, mostly performance-related. Some PRs created by Christopher and Federico
        •  Christopher all my PRs include fixes that are not specifically depending from this release
    • Sandbox store (re-iterated) suggestion: use a high performance IO volume (LHCb is a bit struggling atm with cloud-based not-so-performant volume)
    • py3 client: not advertised yet, a few issues to be sorted out first (connected to LHCb specificities)

    EGI (+ France Grilles):

     Andrei

    • from previous meeting M2Crypto and VOMS extension follow-up (issue)
      • Robot certificates regenerated once again, still to be tested
    • v7r2 deployment on several nodes for testing (EGI has no DIRAC extension, but some users do)
    • Started to use ES (certificate access), WMSHistory is enabled
      • very slow to connect, looking with local manager what’s the issue
      • still, working. So enjoing WMSHistory on ES7.

    Belle2

     Hideki

    • Postponed the migration to v7r0, issue should be sorted and next week deploy hopefully.

    IHEP

     Xiaomei

    • On the way to migrate to v7r1. Plan to use Pilot3
    • CS and WebApp on same machine, upload necessary?
      • it shouldn’t be needed to have 2 different machine
    •  Federico It’s possible to get the Pilot files not from github (which is the default)
    • Some users used the registerFile (from CLI), without checksum … this becomes problematic, does it need to be there?
      •  Daniela we had that problem before. We gave some code to do it programmatically “correctly”. Best solution would probably be to reject registrations without checksum, while adding a “force” flag just in case.
      •  Andrei this is a admin operation, or should be
      • Please create an issue on github!

    DIRAC releases

    • v7r1p41 + v7r1p42:
      • Bugfixes (UserProfileDB, PilotSubmission accounting)
      • Performance improvements for Framework.Monitoring and Public State Manager
    • v7r2p8 + v7r2p9:
      • NEW: (#5122) multi-node allocation support via the ParallelLibraries and srun (slurm)
      • NEW: (#5155) ComponentSupervisionAgent to Monitor running components.
    • v7r3-pre10 (7.3.0a10):
      • CHANGE: (#4715) removing env variable DIRAC_USE_NEWTHREADPOOL (use concurrent.futures instead of DIRAC ThreadPool for agents)

    DIRAC projects

    DIRAC:

    Issues by milestone:

    PRs discussed:

    WebApp:

    •  Federico add VMDIRAC app?
    •  Janusz working on a WebApp for RSS
    •  Andrii In hackathon we found issue for encoding JSON, so this should be fixed before the next hackathon.

    Pilot3:

    • NTR

    DIRACOS:

    • xroot5?
      • NTR
    • condor 9?
      • from previous meeting Overall, HTCondor 9.0 brings many new features but most of them are probably not really useful for DIRAC.
      • no news on integration in DIRACOS (done for DIRACOS2)

    DIRACOS2:

    • from previous meetingfts-rest on py3?
      • NTR
    • from previous meeting Question for rucio client… (issue)?
      • NTR
    • added cachetools
    • oidc-agent? issue opened by  Andrii
      • at the moment not needed, maybe it will be needed in the future. For the moment the issue can be closed.

    VMDIRAC:

    • Agreed to merge into DIRAC for v7r3
      • should be rather easy.  Simon will take care
    • For the Web part a bit more work is necessary.

    Documentation:

    • 1 draft PR open, will be expanded with py3 documentation.

    OAuth2:

    tornado/HTTPs

    • NTR

    management

    • from previous meeting 1 task left for deploying on CVMFS
      • Not yet done  Marko I will come back to it

    COMDIRAC

    • NTR

    other externals, including Rucio

    •  Cedric No progress, tests still to implement/to pass

      •  Federico will contact Janusz and Cedric for technical explanation

    •  Janusz working on Cedric’s code for:

      • making it multi-VO
        • doesn’t look very complex, at least for what regards the SynchronizationAgent
          • testing with 2 VOs in GridPP setup
        • RucioRSSAgent looks a bit more complex (will come next)
      • eliminating the need for Rucio config
        • a few changes to the rucio client itself are needed

    Release planning, tests and certification

    • Certification machine(s)

      • authcertif machine:
        • py3 server installation being tested (the installation procedure is OK)
        • the client will also need to be py3 with a specific code

    • Next hackathon

      • “regular” next week
      • “auth” later (first need to actually fully install it)

    Weekly development(s) focus

    NTR

    AOB and topics from Google forum

    Next hackathon (regular) on June 17th: https://indico.cern.ch/event/1048251/
    Next BiLD on June 24th: https://indico.cern.ch/event/1048250/

    LHCbDIRAC

    • https://trello.com/b/BQvEbteg/deploy-v10r2
    • python3 client on CVMFS
      • Ganga does not work
      • LHCbPilot would need to add a flag
      1. get Ganga to work first
      2. prod for “free users”
      3. for LHCbPilot, we can have a flag and then enable for a subset of the pilots that we send or run
    • SandboxStore: running in certification, looks fine, should be installed in prod
    There are minutes attached to this event. Show them.