Go to the Indico Home Page Oldest event Older event Up to category Newer event Newest event Printable version
 
Focus on:  Hide Contributions
  • Compact style
  • Indico style
  • Indico style - inline minutes
  • Indico style - numbered
  • Indico style - numbered + minutes
  • Indico Weeks View
    • Switch to the management area of this event
    Login

    BiLD-Dev

    → Europe/Zurich
    Federico Stagni (CERN)
    Description

    Bi-Weekly "Loyal" DIRAC developers meeting. And, following, the LHCbDIRAC developers meeting.

    Join Zoom Meeting
    https://cern.zoom.us/j/91083694183?pwd=ZkdDY1I1YkJVc2o3UTdBY1BRZE15UT09

    Meeting ID: 910 8369 4183
    Passcode: 12345678
    One tap mobile
    +41432107108,,91083694183# Switzerland
    +41315280988,,91083694183# Switzerland

    Dial by your location
            +41 43 210 71 08 Switzerland
            +41 31 528 09 88 Switzerland
            +41 43 210 70 42 Switzerland
            +33 1 8699 5831 France
            +33 1 7037 2246 France
            +33 1 7037 9729 France
    Meeting ID: 910 8369 4183
    Find your local number: https://cern.zoom.us/u/artUfAdNB

    Join by SIP
    91083694183@188.184.89.188
    91083694183@188.185.118.153

    Join by H.323
    188.184.89.188
    188.185.118.153
    Meeting ID: 910 8369 4183
    Passcode: 12345678

    working notes
    Hide

    BiLD (Bi-weekly DIRAC Development meeting) – 01/12/2022

    On Zoom: Federico, André, Andrei, Alexandre, Christophe, Christopher, Daniela, Hideki, Simon, Janusz, Xiaomei, Vladimir
    Apologies:

    Follow-up from previous meetings

    • hackathon November 24th, on 8.1.0a5:
      • Few issues found and solved, not specifically related to this release (e.g. for AREX CE). In general rather smooth.
    • Last BiLD 2 weeks ago
      • all updates below

    DIRAC communities roundtable

    LHCb:

    Federico+Alexandre+Christophe+Christopher+Alexey

    • Running in production v7.3.34 (latest v7r3 patch)
    • LHCbDIRAC hackathon done on top of 8.0.5, no issues found in DIRAC
      • not conclusive though
    • applied this change in DFC production and
      • Christophe I would be surprised if anyone else use this
      • Federico should be anyway added to wiki (v8 or no-one will see it)

    EGI

    Andrei

    • struggling to set up a cloud CE (no possibility to specify a project name?)

    ILC/CLIC/FCC/Calice:

    André

    • (Apologies have to leave at 10:30)
    • Still working on server updates to 7.3

    Belle2

    Hideki

    • v7r2 in production
    • User certificate renewal
      • answered, fine with it
    • What is the necessity to migrate from ARC to ARC6?
      • Alexandre maybe you don’t need it, if AREX works fine. ARC6 is an intermediate solution.
      • Daniela WLCG VOs are pushing to move to AREX.
    • from previous meeting Deployed ElasticSearch, issues with ComponentMonitoring
      • Federico ComponentMonitoring on ES never fully worked. What’s working:
        • v7r2: WMSHistory
        • v7r3: WMSHistory and JobParameters
        • v8.0: “everything works” (it is documented), and ComponentMonitoring is spilt between Agent and Service Monitoring.

    Juno

    Xiaomei

    • Condor “WholeNode” parameter not considered?
      • Federico please open Discussion

    GridPP:

    Daniela+Simon+Janusz

    • No changes in Production: v7.3.26
    • Pre-prod: v7.3.34

    Topics from GitHub/Discussions or Google forum

    only un-answered topics below:

    Error in loading WebApp (5.0.1)

    • Should always do pip install WebAppDIRAC[server] ?

    MaxJobsInFillMode replacements

    • this requires a small fix in Pilot

    ERROR: DataString is getting too long

    • DIRAC buffering. The coded “solution” was not accepted, a better one should be coded (suggestion is there)

    Personal certificates with “sn” and “gn” fields not supported?

    • from previous meeting Maybe update to VOMS2CSAgent?

    Pilots/ARC “proxy not found”

    • from previous meeting Andrei we should simply put by the default the bundling of the proxy in the pilots

    …several others in https://github.com/DIRACGrid/DIRAC/discussions?discussions_q=is%3Aunanswered

    from the forum: https://groups.google.com/g/diracgrid-forum/c/l1n_M4ZMDnM/m/K1x0ohbGAwAJ?pli=1

    • Daniela we used/tested this

    DIRAC releases

    v7r3

    • v7.3.34
      • Resources
        • FIX: (#6584) fix delegation process in the AREXCE
        • FIX: (#6539) multi-node allocations with SLURM
        • FIX: (#6535) AREXCE._getDelegation() returns an error if the process cannot be completed

    v8r0

    • v8.0.4
      • test
        • NEW: (#6529) added system/rms_script.sh test for basic RMS test
    • v8.0.5
      • ConfigurationSystem
        • NEW: (#6555) BDII2CSAgent: allow using AREX or ARC6 Computing Elements instead of ARC, fixes #6541

    v8r1

    • v8.1.0a5
      • WorkloadManagementSystem
        • CHANGE: (#6568) improve killing of process with psutil
        • CHANGE: (#6556) completely removed old PilotsLogging machinery
      • DataManagementSystem
        • NEW: (#6567) TornadoS3GatewayHandler service
      • ProductionSystem
        • NEW: (#6567) TornadoProductionManagerHandler service
      • StorageManagementSystem
        • NEW: (#6567) TornadoStorageManagerHandler service
      • MonitoringSystem
        • CHANGE: (#6563) remove setup
      • Core
        • CHANGE: (#6561) Swapped the default of EnableSecurityLogging flag
          • NOTE this effectively makes Elastic/OpenSearch obligatory.
        • CHANGE: (#6466) removed DIRAC_USE_JSON_DECODE, flipped DIRAC_USE_JSON_ENCODE

    DIRAC projects

    DIRAC:

    Issues by milestone:

    Other issues:

    PRs discussed:

    WebApp:

    • Closed few old issues

    Pilot:

    • Closed few old issues

    DIRACOS2:

    • 5 PRs open
    • Chris B Fairly sure I have a fix for htcondor 9.0.x with scitokens
      • Will confirm with Andrei before applying

    Documentation:

    • NTR

    OAuth2:

    tornado/HTTPs

    • from previous meeting Federico for “full production” setup we are not there yet
      • is nginx “mandatory”?
      • can we run more instances?
      • Andrei there’s no “upload” solution for DIRAC SE

    management

    • from previous meeting 3 issues left, still valid
      • Andrei Updated the script, should be uploaded here

    diraccfg

    • from previous meeting Christopher Do we want to make a release that drops support for py2?
      • yes

    COMDIRAC

    • NTR

    DB12

    • NTR

    Rucio

    • NTR

    Tests

    • Federico fixed the submission to SSHBatchCE (code and conf in lbcertifdirac70)

    Release planning, tests and certification

    Certification machines

    • lbcertifdirac70 machine:
      • maybe add an SSHCE site in DIRAC certification submitting to https://batchdocs.web.cern.ch/linuxhpc/index.html cluster ?
        • Alexandre got an access, will be done through diracgri account
      • Computing Element of each supported type should be configured (HTCondorCE, 2 ARCS (Alexandre (maybe use RALPP and Glasgow ? I can talk to them if there are issues --Daniela)), 1 Cloud (Daniela - this is already part of the tests), SSH). Consider OAuth tests as well - what is needed as far as certification servers configuration is concerned.
      • Daniela: Enabled (and tested, at least with X509) WLCG VO for token testing. Needs update in puppet for /etc/grid-security/vomsdir and /etc/vomses, please.
        • Christophe I’ll check and add it
    • lbcertifdiracoauth machine:
      • NTR

    Next hackathon(s)

    • Next week, on lbcertifidirac70 and with v8.1.0aX

    AOB

    CHEP2023

    • abstracts submitted by Federico (generic), Daniela+Simon+Janusz, Alexandre (HPC, LHCb-ish), Xiaomi one for Juno, Andrei one for CTADIRAC, Christopher one LHCb-ish for TS

    [ISGC] Andrei abstract for AuthN/Z

    Confirmed next workshop as “DIRAC&Rucio workshop 2023”, KEK, 16-20 October 2023

    • added to HSF calendar
    • no indico yet

    Next hackathon: December 8th
    Next BiLD: December 15th

    LHCbDIRAC

    • v10r4: deploy board in https://trello.com/b/kzUKdMts/deploy-v10r3
      • added note for HTTPs CS master deploy
      • DFC update scheduled for next Wed
        • for how long? message to users?
    • https://lhcb-auth.web.cern.ch/
      • from previous meeting Andrei this is not properly configured yet. Not usable for Pilot submission yet (compute scopes not added).
    • Federico For the certification setup: I have set again dryRun=False for Bdii2CSAgent
    • LHCbDIRAC hackathon
      • when? Next Tuesday OK?
    • Move from Jira to GitLab for issues: CERN won’t renew the contract with Atlassian/Jira at the current conditions, and if somehow Jira will stay the feeling is that will be only temporary.
      • Anyone would be against to move to using GitLab issues instead?
      • For the existing ones: review, close if you can, or keep it to be moved for later.
      • Jira history: not sure if it will be possible to keep it.
    There are minutes attached to this event. Show them.
      • 10:00 → 10:10
        Items from Previous BiLD-Dev 10m
      • 10:10 → 10:20
        DIRAC Communities roundtable 10m
      • 10:20 → 10:30
        DIRAC releases 10m
      • 10:30 → 10:55
        DIRAC projects 25m
        • DIRAC
        • WebApp
        • Pilot
        • DIRACOS
        • DIRACOS2
        • VMDIRAC
        • Documentation
        • OAuth2
        • tornado/HTTPs
        • other externals (include Rucio)
        DIRAC certification
      • 10:55 → 11:00
        Release planning, tests and certification 5m
        DIRAC PRs
        google forum
      • 11:00 → 11:15
        Weekly development(s) focus 15m
      • 11:15 → 11:25
        AOB
        Convener: Federico Stagni (CERN)
      • 11:25 → 11:40
        LHCbDIRAC 15m