Focus on:  Hide Contributions
Login

BiLD-Dev

→ Europe/Zurich
2/R-014 (CERN)

2/R-014

CERN

10
Show room on map
Federico Stagni (CERN)
Description

Bi-Weekly "Loyal" DIRAC developers meeting. And, following, the LHCbDIRAC developers meeting.

Zoom: BiLD
https://cern.zoom.us/j/62504856418?pwd=TU1kb01SOFFpSDBJeWVBdU9qemVXQT09

Meeting ID: 62504856418
Passcode: 12345678
 

Live notes
BiLD
Zoom Meeting ID
62504856418
Host
Federico Stagni
Useful links
Join via phone
Zoom URL
Hide

BiLD – 18/12/2025

At CERN: Federico, Christophe (later), Christopher, Ryun, Alexandre
On Zoom: Andrei, Daniela, Hideki, Alexei, Mazen, Xiaomei, Simon
Apologies:

Previous meetings

  • Last BiLD was 3 weeks ago
  • 1 week ago we planned a certification hackathon, that effectively did not happen as we started discussing the merging strategy (main topic for today, see below)

DIRAC communities roundtable

LHCb:

 Federico+Christopher+Christophe+Alexandre+Ryun

  • NTR
  • 550K running jobs record :chart_with_upwards_trend: :champagne:

Belle2

 Hideki, Ueda

  • Records are sent both in accounting and monitoring
  • Also asked about future, pointed to DiracX ticket

Juno+BES3:

 Xiaomei

  • from previous meeting Wants to clean VOMS- discussion to be open
  • from previous meeting Forced to use Multi-hop. Mostly successfull, but issues reported
    • PR 8386 has been created, to be tried.
      • for 9.0

EGI

 Andrei, Mazen

  • Fighthing for pilots’ acceptance by site
  • Setting up new service for IN2P3 supported VOs
  • from previous meeting Created GreenDIRAC extension for keeping related developments: GreenReportingAgent, energy consumption by jobs tool (Mazen)
    • Submitting energy metrics for each job into the DB
    • investigating the data obtained, also for its visualization. Data kept in OpenSearch together with the Job Parameters
  • from previous meeting Started a GreenSiteDirector
    • work in progress, for submitting to sites that have a lower energy footprint
  • from previous meeting Got functional D9/Dx0.0.1 certification installation (Bertrand):
    • DIRAC9 host + 2 hosts k3s cluster for DiracX + S3/minIO
      •  Chris+Federico should not run on 2. Better 1 or 3
    • Opensearch with certificate authentication
      • will contribute to DiracX for this
    • IAM dev IdP, failed with dteam IAM, Check-In in progress
    • wants to add also a MQ

GridPP:

 Daniela, Simon

  • NTR

Topics from GitHub discussions and bots

Releases

DIRAC

  • v8.0.77
    • still the last
  • v9.0.14
    • NEW: (#8382) Attempt to set the xrd checksum attribute when copying files localy
      • needs the latest DIRACOS release
    • NEW: (#8392) handle adding user AffiliationEnds section in the CS when modifying the user from CSAPI

Issues (still there?):

DiracX

DIRAC merging strategy

At the previous hackathon, we ended up discussing https://codimd.web.cern.ch/CKFZsVZ9TSy40sUiqGZV0w which contained few notes about: what to merge, what to test in certif, how to go forward in general.

GitFlow vs TBD

We used GitFlow (or sort-of) for several years

  • the integration branch has been our develop branch
  • creating pre-releases from integration, and releases from rel-vXrY branch
  • using “sweeper” for creating PRs for “upper” branches
  • pros:
    • clear separation between pre- and production releases
  • cons:
    • slow pacing

TBD just means “no branching”, so basically there’s only the main (for us the integration) branch. Question is if this can work for us. From https://medium.com/yield-studio/trunk-based-development-tbd-vs-git-flow-b73bb110452d:

With TBD, this means that all code modifications are introduced and merged directly into the main branch. Each developer must therefore merge their changes regularly, several times a day.
… code reviews should be handled quickly (within half a day)
TBD requires rigorous testing, vastly automated.

Proposals

Given that:

  • for client, server and pilot we have always the option of installing a branch, instead of a (pre)release
    • this has been recently added to SystemAdministrator via PR 8403 + PR 8405, for the pilot it is there since long time
  • we distinguish between “easy fix PRs” and “risky PRs”:
    • we can merge/release “easy fix PRs”
    • “risky PRs” have the need to be tested in the certification setup

3 options for TBD adoption:

1. The “dirty” integration branch

  • always install integration in certification

pros:

  • easy and strightforward

cons:

  • we can’t make a release until we fixed the integration branch, by fixing the code introduced by the risky PR, or by reverting it

2. The certif branch

  • branch off a temporary certif branch that will be installed in certification

pros:

  • we can create fix releases while keeping the integration branch clean

cons:

  • more combersome
  • The certif branch needs to be kept up-to-date

3. The PR branch

  • install un updated PR branch in certification (git+https://github.com/fstagni/DIRAC.git@my_risky_PR)

pros:

  • we can create fix releases while keeping the integration branch clean (as per option 2)
  • still easy, but for extensions needs care
    cons:
  • one PR at a time (probably OK)
  • the PR branch needs to be kept up-to-date (as per option 2)

Considerations

In all cases:

  • we can (should) rename integration to main
  • there are no more pre-releases (ever!)
  • the pilot that we run in certification will effectively always install DIRAC (not setting up DIRAC from CVMFS)
  • we can create only patches of v9.0 from now on, but we also can create minor releases, and in this case, we either:
    • “maintain” the minor releses (this would require cherry-picking going backward)
    • once we created, say, v9.1, there’s no more v9.0 patching (much simpler, and more “semantic”)
  • we should avoid creating PRs with breaking changes, meaning that DB changes should be behind settings – developers work!
  • if we go for TBD, we are putting in question the certification hackathons (which become much more developer-centric)

Conclusion

  • There is general support for option 3 (use the the PR branch)
  • The same thing can also for be done for diracx
    • instructions to update diracx to branch “inside” DIRAC would need to be provided

Release planning, tests and certification

  • Upgrade to v9+0.0.1:

    • No news

  • Certification machines

    • from previous meeting  Federico added the diracx_admin VO. Somehow not working fully …?

  • Next hackathon(s)

    • not planned

DIRAC projects

DIRAC:

Issues by milestone:

Other issues:

  • Get infos from CRIC
    • any volunteer?
  • Replacement for BDII2CSAgent #8194
    • 13th Nov New updates in https://helpdesk.ggus.eu/#ticket/zoom/3427
      •  Andrè (enthusiastically?) volunteered ;-)
      • GOCDB has agreed to implement the scopes as we originally asked for.
      • As a test, we implented scopes for ilc and t2k.org, that the sites can set for their services.
      • Once Andre S has finished his DIRAC side tests, we can submit a GGUS ticket and they will generated scopes for all the VOs we give them.
      • Then we just need to find a way to diplomatically phrase an EGI broadcast, so we hopefully don’t have to contact all sites individually.

PRs discussed:

WebApp:

  • NTR

Pilot:

  • PR for removing Python2 support re-created (previous one introduced a bug)
    • Tested in hackathon, will be merged into master when all tests pass (pending RAL support @@)
    • from previous meeting if you really want to keep py2 support, there’a tag

DIRACOS:

Documentation:

management

  • Having again issues for /cvmfs/dirac.egi.eu (the syncing was not happening, ticketed, etc.). No monitoring on their side, urgent need to move out.
  • from previous meeting new /cvmfs/dirac.cern.ch repository created – CERN ticket
    • action on  cburr to populate it (using LHCb “machinery”)

DB12

  • NTR
  • from previous meeting  Federico “I asked  Igor if he wanted to become one if not the main maintainer”
  • from previous meeting We should agree on a strategy on how to do things here, as PRs can’t just be merged:
    • there are reports created that depend from it, merging/releasing on a random Tuesday is not the way to go
    • can we have parallel benchmarks? not clear actually if that’s doable
    •  Federico maybe create a “2026” release?
      • we should see if there would be time for that

Rucio

Tests

  • NTR

DiracX:

Issues and Discussions

PRs discussed:

CWL prototype

DiracX-charts:

DiracX-web:

  • NTR, but many “bumps” PRs

Next appointments

AOB

LHCbDIRAC

There are minutes attached to this event. Show them.
    • 10:00 → 10:10
      Items from Previous BiLD-Dev 10m
    • 10:10 → 10:20
      DIRAC Communities roundtable 10m
    • 10:20 → 10:30
      DIRAC releases 10m
    • 10:30 → 10:55
      DIRAC projects 25m
      • DIRAC
      • WebApp
      • Pilot
      • DIRACOS2
      • VMDIRAC
      • Documentation
      • OAuth2
      • DiracX
      • other externals (include Rucio)
    • 10:55 → 11:00
      Release planning, tests and certification 5m
    • 11:00 → 11:15
      Weekly development(s) focus 15m
    • 11:15 → 11:25
      AOB
      Convener: Federico Stagni (CERN)
    • 11:25 → 11:40
      LHCbDIRAC 15m
Powered by Indico v3.3.10-pre