Go to the Indico Home Page Oldest event Older event Up to category Newer event Newest event Printable version
 
Focus on:  Hide Contributions
  • Compact style
  • Indico style
  • Indico style - inline minutes
  • Indico style - numbered
  • Indico style - numbered + minutes
  • Indico Weeks View
    • Switch to the management area of this event
    Login

    BiLD-Dev

    → Europe/Zurich
    Federico Stagni (CERN)
    Description

    Bi-Weekly "Loyal" DIRAC developers meeting. And, following, the LHCbDIRAC developers meeting.

    Join Zoom Meeting
    https://cern.zoom.us/j/91083694183?pwd=ZkdDY1I1YkJVc2o3UTdBY1BRZE15UT09

    Meeting ID: 910 8369 4183
    Passcode: 12345678
    One tap mobile
    +41432107108,,91083694183# Switzerland
    +41315280988,,91083694183# Switzerland

    Dial by your location
            +41 43 210 71 08 Switzerland
            +41 31 528 09 88 Switzerland
            +41 43 210 70 42 Switzerland
            +33 1 8699 5831 France
            +33 1 7037 2246 France
            +33 1 7037 9729 France
    Meeting ID: 910 8369 4183
    Find your local number: https://cern.zoom.us/u/artUfAdNB

    Join by SIP
    91083694183@188.184.89.188
    91083694183@188.185.118.153

    Join by H.323
    188.184.89.188
    188.185.118.153
    Meeting ID: 910 8369 4183
    Passcode: 12345678

    working notes
    Hide

    BiLD (Bi-weekly DIRAC Development meeting) – 08/07/2021

    At CERN: Still, nobody :/
    On Zoom: Federico, Abderahaman (LUPM new collaborator), Alexandre, Andrii, Christophe, Christopher, Daniela, Hideki, Luisa, Janusz, Simon, Ueda, Xiaomei
    Apologies: Andrei

    Follow-up from previous meetings

    • Ran mini hackathon last week on py3 DIRAC:
    • PRs merging strategy?
      • this is done, but some discussions emerged so not yet applied (see PR). It would be about time to conclude
        • “parallel” discussion on using black for formatting
          • Q: is it really necessary?

    DIRAC communities roundtable

    GridPP:

     Daniela

    • Running v7r1p39
    • v7r2 on the pre-production server

    LHCb:

     Federico+Christophe+Christopher

    • NTR

    Belle2

     Hideki+Ueda

    • DBs extension: TaskQueueDB, TransformationDB
      • TransformationDB and Rucio integration likely problematic? Will open an issue on GitHub

    IHEP

     Xiaomei

    CTA

     Luisa

    • Instance distributed between IN2P3 and PIC
    • MySQL transfer (MariaDB 10.5)
      • GROUP BY statements change
      • PR will need to be make, will be backward compatible
    • Running v7r2p11
      • tested “almost” everything
    • Asked by the ESCAPE project to participate in the next DC, which is using Rucio
      • asking help from Belle colleagues, in August
      • Rucio version in DIRACOS==1.25.5

    DIRAC releases

    • v7r1p44:
    • v7r2p12 + v7r2p13:
      • NEW: (#5212) SiteDirector: added flag for pilots to install python3 DIRAC clients
      • CHANGE: (#5219) Service Handlers discover DBs in Extensions
        • which is not a full solution (see discussion in PR)
        • Several other fixes, some for py3
    • v7r3-pre13 (7.3.0a13) + v7r3-pre14 (7.3.0a14):
      • Several py3 fixes (installation + actual code)

    DIRAC projects

    DIRAC:

    Issues by milestone:

    PRs discussed:

    WebApp:

    • Several PRs open+reviewed+merged for py3 (release 4.3)
    • 4.3.0a9 on pypi for py3: https://pypi.org/project/WebAppDIRAC/
    • To reach there we:
      • Created dirac-webapp-packaging (on github and on pypi)so that WebApps can be automatically built during packaging for PyPI or installation with pip
      • Created DIRACWebAppResources (on github and on pypi) to distribute the ExtJS SDK to reduce the size of WebAppDIRAC releases
      • The static directories served by nginx are now symlinked to /WebApp/StaticResourceLinkDir while launching the WebApp so the path is predictable
      • Reflected these changes in the puppet profile (unmerged still)
    •  Andrii wants to use the same tornado service for DIRAC and WebApp
      • as of now being tested on authcertif

    Pilot:

    DIRACOS:

    • xroot5?
      • NTR
    • condor 9?
      • NTR

    DIRACOS2:

    • from previous meetingfts-rest on py3?
      • NTR
    • from previous meeting  Andrii jwt + other token stuff need to be added here
      • Not yet done
    • During the hackthon found that an issue on how arc was built, new release soon
      • would be useful to also test HTCondor (should be OK, given we ran jobs in oauthcertif setup)

    VMDIRAC:

    • from previous meeting Agreed to merge into DIRAC for v7r3
      • PR created: it has been reviewed, and moved back to draft

    Documentation:

    • NTR

    OAuth2:

    tornado/HTTPs

    management

    • from previous meeting 1 task left for deploying on CVMFS
      • Not yet done, Marko left, Andrei should take over
    • 3 PRs (#40, #41, #42) created and merged for py3
      • Also for creating WebApp releases

    diraccfg

    • Release 0.2.2 created, added a sanity check of the CFG
      • and spotted bugs (in 2 tests) following that – PRs for v7r2 created+merged+propagated

    COMDIRAC

    • NTR

    DB12

    • (forgot to mention in the meeting… )
    • Summer student (LHCb) developing the DB12 package:
      • py2/py3
      • adding tests, github actions
      • publishing on pypi
      • using it in Vanilla DIRAC (to do)
      • running tests for different MP + other architectures (later)

    other externals, including Rucio

    • Issue: [URGENT] Rucio MUST be an optional dependency (#5250)
      • LHCb developers keep fixing rucio integration issues and that’s not cool
      •  Federico it would be enough to have proper responsibility by someone of those that are interested
      •  Christophe an optional dependency would make sure we don’t need to think about it
      •  Ueda we have it in BelleDIRAC extension anyway
      • not found any reason for NOT making a RucioDIRAC extension
      • Cedric’s not present to the meeting making it a bit difficult to take final decision
    •  Janusz working for making rucio-DIRAC multi-VO implementation, but most importantly seeing if there’s really the need for a rucio configuration file (it should not).

    Release planning, tests and certification

    • Certification machines

      • lbcertifdirac70 machine:
        • Did not touch it lately
      • lbcertifdiracoath machine:
        • installed py3 server
        • Should move from VOMS to IaM

    • Next hackathon(s)

      • On July 15th (who’s not there?)
        •  Federico We can go back to lbcertifdirac70 and run on py2, do a “standard” hackathon
        • For the hackathon in 3 weeks we can move lbcertifdirac70 to py3
          • flick back and forth py2/py3? possible but not advisable
      • We might also call a mini-hackathon on oauth/token

    AOB and topics from Google forum

    Next hackathon (py2) on July 15th: https://indico.cern.ch/event/1056182/
    Next BiLD on July 22nd: https://indico.cern.ch/event/1057796/

    LHCbDIRAC

    • https://trello.com/b/BQvEbteg/deploy-v10r2
      • No changes in the last 2 weeks
      • We first need to install a new release, but for that we’ll first need a DIRAC release
        • Won’t install LHCbDIRAC release before Wednesday
      • We need new boxes + duplicate the JobManager
    • python3 client on CVMFS
      • Ganga and PID-calib using it
      •  Christophe keep bumping in division errors
      • after release we can make it the default, also for the pilot
    • SandboxStore: installed in prod
      • 46% migrated - rsync-ing
      • disabled the cleaning
    • we should start LHCbDIRAC hackathon based on v7r3/py2 and later py3
      • We’ll need to move server by server
      • the BKK might need some updates (it’s very delicate)
    There are minutes attached to this event. Show them.