Go to the Indico Home Page Oldest event Older event Up to category Newer event Newest event Printable version
 
Focus on:  Hide Contributions
  • Compact style
  • Indico style
  • Indico style - inline minutes
  • Indico style - numbered
  • Indico style - numbered + minutes
  • Indico Weeks View
    • Switch to the management area of this event
    Login

    BiLD-Dev

    → Europe/Zurich
    Federico Stagni (Conseil Europeen Recherche Nucl. (CERN)-Unknown-Unknown)
    Description

    Bi-Weekly "Loyal" DIRAC developers meeting. And, following, the LHCbDIRAC developers meeting.

    Join Zoom Meeting
    https://cern.zoom.us/j/91083694183?pwd=ZkdDY1I1YkJVc2o3UTdBY1BRZE15UT09

    Meeting ID: 910 8369 4183
    Passcode: 12345678
    One tap mobile
    +41432107108,,91083694183# Switzerland
    +41315280988,,91083694183# Switzerland

    Dial by your location
            +41 43 210 71 08 Switzerland
            +41 31 528 09 88 Switzerland
            +41 43 210 70 42 Switzerland
            +33 1 8699 5831 France
            +33 1 7037 2246 France
            +33 1 7037 9729 France
    Meeting ID: 910 8369 4183
    Find your local number: https://cern.zoom.us/u/artUfAdNB

    Join by SIP
    91083694183@188.184.89.188
    91083694183@188.185.118.153

    Join by H.323
    188.184.89.188
    188.185.118.153
    Meeting ID: 910 8369 4183
    Passcode: 12345678

    working notes
    Hide

    BiLD (Bi-weekly DIRAC Development meeting) – 11/02/2021

    At CERN: Still, nobody :/
    On Zoom: Federico, Alexandre, André, Andrei, Andrii, Christophe, Christopher, Daniela, Hideki, Marko, Simon, Vladimir
    Apologies: Cedric

    Follow-up from previous meeting

    • Ran hackathon last week on v7r2-pre32:
      • trello board here
      • One issue with a script used by the pilots made it such that we could not test many things
        • We fixed everything that was “not passing”
      • VMDIRAC
        • added to instructions
        • dirac-installed but no services/agents running
        • Need configuration
      • https services tested (DFC on https)
        • issues found, PR with fix

    DIRAC communities roundtable

    GridPP:

     Daniela

    • Upgraded to v7r1p26. No problems observed.

    CLIC:

     André

    • NTR

    LHCb:

     Federico

    • MP queues, mostly from Arc CEs. Still a few issues being addressed.
    • Switched to Singularity InnerCE at one CE at CERN and ended up with issues (image could not be found in CVMFS). Not clear why, opened PR for debugging
      • On a somewhat related note: through the LHCb Pilot we are adding tags for each CVMFS location mounted, and use that for matching jobs. Can be backported to vanilla Pilot if there’s an interest.

    EGI:

     Andrei

    • Slowly preparing the migration of the services. WebPortal already done.

    France Grilles:

     Andrei

    • NTR

    Belle2:

     Hideki

    • Started migration to v7r0. Not yet in certification, but installed “locally”. Have not seen issues up to now.
    • Security inspector: complain that M2Crypto server still accepts TLSv1. Check if can be disabled.
      •  Christophe we would need everyone to be on v7r1
      • DIRAC Issue will be re-opened
    • Rucio: few struggles with APIs, new version of rucio will be

    DIRAC releases

    • v7r0p48:
    • v7r1p28:
      • CHANGE: Revert #4919 - absolute imports in several modules
      • CHANGE: (#4926) JobAgent: flipped the default for fillingMode
      • CHANGE: (#4926) SiteDirector: The max number of jobs in filling mode is not set anymore by the SiteDirector. The option MaxJobsInFillMode was removed
    • v7r2-pre33:

    DIRAC projects

    DIRAC:

    PRs

    • Fixes for scripts and docs that were not propageted from v7r1: https://github.com/DIRACGrid/DIRAC/pull/4958
      • better documentation from DIRAC scripts
      • suggestion is to merge ~quickly. Federico and Christopher looked into it, seems fine. Andre’ will also look.
      •  Andrii wants to also register arguments not only options
        •  Federico please do it another PR once this is merged.

    Issues

    • v7r0:
      • only 1 issue left
    • v7r1:
      • almost only documentation needed
    • v7r2:
      • discussed at previous meeting
    • v7r3:
      • Not discussed

    WebApp:

    • python3 and WebApp changes: to be done, should target branch v4r2

    PRs

    • 2 PRs open. Looks like gitlab-CI workflows are broken.
      • json linter broken?  Andrii
      •  Christopher the workflow is just broken, should copy from LHCbWebAppDIRAC (issue opened).

    Issues

    • 4 issues opened in the last 2 months, all by Philippe.
      • Mostly for the plotting of the accounting
      •  Christophe the plotting system would probably need to be fully replaced. There’s an issue in DIRAC for that: https://github.com/DIRACGrid/DIRAC/issues/4574
        • won’t discuss today but please collect there your thoughts and requirements.

    Pilot3:

    DIRACOS:

    • New DIRACOS release v1r17
      • new gfal2 version + davix
        • gfal2: last release candidate
        • davix: using a development version.  Chris used for testing HTTPs endpoints, working fine.
      • adds rucio
      • package cachetools was removed (was only used by LHCb, which added it to LHCbDIRACOS)
      • LHCb using v1r17 in production and working fine.
    •  Hideki can we update the packages of DIRACOS in our extension (need to do it for latest rucio)?
      •  Christophe sure, but you can do it in vanilla DIRACOS for everyone to benefit
      •  Federico Belle2 is using v6r22, who is still on DIRACOS v1r3 (very old). We can’t fully say that bumping to lastest DIRACOS would come without issue.

    DIRACOS2:

    • Still waiting for fts-rest on py3
      • delayed that
    • When installing python 3, we are still installing from Chris B fork?

    Reminders:

    • DIRACOS2 is for python3 only. No efforts will be ever put into python2.
    • It will be offered for client installations only starting from DIRAC v7r2
      • Future versions should flip the flag of the client, and offer a server python3 installations (which right now is broken, won’t look for fixing it in v7r2)
      • for v7r3 we maybe make it as default for client installations

    VMDIRAC:

    • Maybe some dependencies are not in DIRACOS, like libcloud. With certification we’ll find out.
    • python3?
      • it’s through the py3 compatibility test
      •  Christopher worth running py3 pylint directly.

    Documentation:

    • NTR

    OAuth2:

    • WIP
    • In issue WLCG and Token transition – reminders and requirements there are a few links of meetings and docs that WLCG is doing. As DIRAC we should be present there (DIRAC is also mentioned). Now that we are approaching DIRAC v7r3 we need to make sure that what’s done is complying with the requirements being discussed at WLCG level.
      •  Andrei I will try to be present there

    tornado/HTTPs

    • Couple fixes added after last hackathon.

    management

    diraccfg

    • NTR

    other externals, including Rucio

    •  Daniela
    • Follow up wt Janusz’ email and Federico’s comment. Imperial is under immense political (and funding) pressure to deliver (as in “released”) something for DIRAC-RUCIO that is multi-VO aware very soon, even if it is a minimal implementation and we have to hack it into shape later.
    • For everyone else, here are the pertinent points from Janusz’ proposal:
      • Since it has been decided that Belle’s implementation of a Rucio file catalog is the way to go we would like to start work on making it VO-aware and integrate it with our Multi-VO Rucio server at RAL.
      • The current RucioFileCatalogClient init() signature is flexible enough, so we would come up with a small PR showing how we could eliminate the need of the Rucio config file or any env vars and give preferences to arguments passed into the Rucio client() call. The current PR gives us no chance to do it since the Rucio client() is called w/o the arguments and passes the proxy location to the client() in the environment.
      • In the original client() implementation, which is a part of Rucio API the input parameters take precedence over other configuration options, so once we provide those in Dirac config we should be ready to go. A list of allowed arguments in the Rucio client() call looks sufficient for a generic integration w/o a config file. If the input arguments are not provided by Dirac, they would be provided by the config file - Belle way.
    • As mentioned in the introduction, Imperial is asking for an imminent integration of the Belle code into code DIRAC, so we can start adapting it.
    •  Federico: There are two existing PRs:
    • (Federico cont’): From the report that Cedric gave at the previous BiLD meeting (https://demo.hedgedoc.org/KnAARicoTsi-8Cr-V0NFvQ#Belle2) it was not so clear if Belle’s implementation if fully shareable. I would anyway ask Cedric to put in DIRAC, even if it contains specific Belle’s code. The target branch does not really matter, it can even be v6r22, it does not need to be v7r2.
    • Janusz would prefer to develop against v7r1 as this is what Imperial has set up everywhere.
    • Cedric is implementing the suggestions and updating the PR today.

    Release planning, tests and certification

    • Certification machine

      • VMDIRAC
        •  Daniela I am working on it - need some input from Federico (appcredential/“software not installed error”)
          • I am updating the documentation as I go along.
      • Reminder: task for multi-vo

    • Next hackathon

      • next week (18th Feb)
        •  Federico Most probably I won’t be there, I will prepare for the hackathon to happen but you do it without me.

    Weekly development(s) focus

    PRs merging strategy

    Issue opened: https://github.com/DIRACGrid/DIRAC/issues/4941

    HTCondorCE submission and usage

    • Issue opened https://github.com/DIRACGrid/DIRAC/issues/4935 by GridKA admins in response to LHCb’s “misusage” (in GridKA’s terms). Basically: since we are not running a local schedd, the jobs (the pilots) stay in “completed” and don’t get removed. After 100K accumulates, pilots submission halts.
      • in GGUS ticket we discussed an operational solution. It will be tested and maybe added as possibility when local schedd is not used. Maybe it’s just a matter of changing the defaults.
    •  Federico I sent around a mail asked basically what to do:
      1. run our own schedd
      2. do something different, but with the net result of basically “download all your pilot outputs as soon as you are done”: development in DIRAC that, independently of what’s the CE type (HtCondor, Arc, SSH, vac, cloud etc), would store 100% of pilots outputs somewhere for some time. This would be nice but I don’t know if anyone would have time to develop that. Certainly not tomorrow, and not without planning.
        • there’s an option to store the pilot output as blobs in MySQL, but this is not scalable. This is used for caching as of now.
        • the SiteDirector has an option for that already but not used as of now
        • ideally you would store the pilot outputs in ElasticSearch and do search in it.
    •  Daniela GridPP does not care too much, as long as you leave the current system in place and don’t make the running of a local schedd compulsory - so far we had no complaints. Also: The pilots on our (Imperial’s) HTCondorCEs show as “Done” - is that different to “Completed” ? Shouldn’t the pilot logs get auto deleted after a limited time anyway, i.e. when the pilots get removed ? I (Daniela, because Simon might know/remember more than me :-S ) thought that was the case anyway? We rather not download the pilot logs unless we have to and we rarely if ever look at logs that are more than a day old.
    • CLIC runs one schedd for all the sites. Looks like running without much issues. It downloads all outputs on the file system, gets removed periodically.
    •  Chris B A naive estimate of the pilot log size in LHCb would be 180GB a week (1.9 million files, 1.2 billion lines, easily compressed to 30GB)

    AOB and topics from Google forum

    Next hackathon on February 18th: https://indico.cern.ch/event/1008132/
    Next BiLD on February 25th: https://indico.cern.ch/event/1008133/

    DUW10 online: https://indico.cern.ch/event/852597/overview 10-11th May 2021.
    will announce it in 2 weeks if there are no further comments.

    LHCbDIRAC

    There are minutes attached to this event. Show them.