Choose timezone
Your profile timezone:
BiLD-Dev
Description
Bi-Weekly "Loyal" DIRAC developers meeting.
And, following, the LHCbDIRAC developers meeting.
Bi-weekly DIRAC Development meeting -- 07/03/2019
At CERN: Federico, Zoltan, Christophe, Marko, Andre', Sasha
On Vidyo: Andrei, Daniela, Hideki, Simon, Xiaomei, Marcelo, Vladimir, Andrii, Wojciech
Apologies:
Follow-up from previous meeting
- Move of web server: the last file used from AFS (globalDefaults.cfg) will not be used anymore starting from DIRAC v6r21
- the old one will die together with AFS (end of LHC LS2) so by that time everyone should move to v6r21+
DIRAC communities roundtable
- CLIC:
- Scaling up (many MC jobs)
- some services duplicated (TransformationManager, DataStore (helper + master))
- Exectors chains also duplicated
- Suggestion: look in https://dirac.readthedocs.io/en/latest/AdministratorGuide/scalingAndLimitations.html#scaling for info on scaling up
- This guide would need to be expanded
- needs v6r21
- LHCb:
- Will use v6r21 in production ASAP (next week)
- GridPP:
- Stager (AKA StorageManagementSystem) -- tried to use it for a multi-VO installation but it's only single VO
- will look for making it multi-VO
- Chris: at some point we'll replace this system with RMS + FTS3, but no deadline yet
- in the meantime if there's only one VO that needs it can be used already now
- will look for making it multi-VO
- Stager (AKA StorageManagementSystem) -- tried to use it for a multi-VO installation but it's only single VO
- Belle2:
- Still running very old v6r17
- going to upgrade soon to v6r20?
- ...time is ticking!
- going to upgrade soon to v6r20?
- Still running very old v6r17
- Juno:
- FTS3+TS+RMS being tried out for DDM system
- VMDIRAC: cloud director gets stuck when trying to connect to one cloud (using deprecated libcloud)
- need new version
- IGI:
- Just upgraded to v6r20p26
- Check in the matcher for pilot version can probably be skipped (especially for Multi-VO)
- One more experiment joining
- EGI:
- Smooth operation
- Working for DIRAC communities, including cloud sites
- Ongoing development with the EGI check-in and authentication in the portal. Being tested now.
- dirac-proxy-init gives you a URL that's passed to browser where the SSO happens.
- Working also on using "DIRAC CA" for internal-only authentication
- Has to consider the changes in M2Crypto PR
- Smooth operation
- [Related] ESCAPE:
- new project H2020 (mostly for astrophysics)
- Marseille will have one person for km3net for setting up their computing
- [Related] DIRAC-RUCIO integration
- pressure from several users
- Current DIRAC developers won't do the job but other interested partners might (Belle2, SKA, CTA, ...)
Current situation
DIRAC
- v6r20p26 still the last one
- v6r21:
- compiled today -- follow instructions in wiki for FTS DB update
VM:
- libcloud being obsoleted
- new connector for OpenStack
- occi connect becoming obsolete too
- can use VM in MP mode (not multi-pilot)
WebApp:
- NTR
Pilot3:
- Wojtek: PR for re-factorization of pilot logger. Hiding the way we do the logging.
- Need to add functionality to replace "request" because it's not available for python 2.6, need to use urllib
- Chris: in python 2.6 is impossible to connect to HTTPS, would need to use system calls (urllib doesn't work)
- Daniel adapting part of the code using tornado (for syncing DNs from CS).
- In few weeks it will be hopefully ready
DIRACOS:
- v6r21 can install DIRACOS, but then updates wouldn't work from the SystemAdministrator (for that we would need to wait for v7r0)
- A howto for doing the upgrade will be prepared
Documentation
- Big PR from Andre': https://github.com/DIRACGrid/DIRAC/pull/3980
- anchors
- automatically created documentation files
- error codes documented
- No updates to website
BOINC:
- NTR
DB12:
- NTR
tornado and other externals
- Large PR for M2Crypto created. Have to be re-reviewed carefully, i.e. by Simon
- could not do an integration test for the executors, need to be checked carefully in certification
Release planning, tests and certification
Release planning:
- v6r20 series
- v6r20p26 still the last patch of v6r20
- v6r21 series
- created today, will be announced shortly
- v7r0 series
- Many BIG PRs, need to merge in a certain order, rebase, etc.
- Should install on DIRAC certification with:
- DIRAC v7r0-preX
- DIRACOS v1r0
- WebApp 4.0
- releases.cfg should be modified accordingly [done?]
- bashrc will need to be re-done, so a fresh installation is needed
- Instructions should go to https://github.com/DIRACGrid/DIRAC/wiki/DIRAC-v7r0
- All Jenkins tests should be updated
Certification process:
- The DIRAC certification machine should be re-done from scratch
- Federico created new DB, still to be configured
- should use CentOS7
- Jenkins pipelines:
- working fine for LHCbPilot
Weekly development(s) focus
NTR
DIRAC: current PRs and tasks being worked on, or topics from Google forum
- NTR
AOB
DIRAC poster ready, can be downloaded and printed and attached in your institute! https://drive.google.com/file/d/1L0_moK5WZkM7phS7jjxwHY6gocmMIIj1/view
---------------------------------------------------------------------------------------------------------------
LHCb
- Create last pre-release of v9r3 then last tests from Federico plus go in production next week.
- Use DIRACOS in pilots for v9r4?
- Re-organization of ES and Kibana will start to be a topic
There are minutes attached to this event.
Show them.
