Choose timezone
Your profile timezone:
BiLD meeting
→
Europe/Zurich
2-R-014 (CERN)
2-R-014
CERN
Description
Dial-in numbers: +41227676000 (English-US, Main)
Access codes: 0146874 (Leader)
0135655 (Participant)
Participant site: https://audioconf.cern.ch/call/0135655
Bild:
Present : Adri, Federico, Elisa, Vladimir, Mario, Andrei, Matvey, Zoltan, Joel, Ricardo, Krys, Stefan, Antonio
Excused : Philippe
Previous minutes:
DEV and CERT proposal : accepted
DIRAC, DiracWeb :
v6r2 : advanced test for transfer done in Online. So far tests are OK. StorageUsage faster with the new Catalog. Should include a pull to take more than one file catalog in in the API. Release date : pre-release tomorrow and Release next week.
Diracweb : profile for presenter : work in progress
Is it possible to share presenter page ? code is in the pipeline
Jobscheduling should go to v6r3 series.
Other VO : City, SuperB testing of Dirac against Amga. Dirac services installed in Asian region.
2 students working on Boinc and National Chinese Grid.
PIC tested cernvm image on some cloud. (VMDIRAC project)
LHCbDirac, LHCbDiracWeb:
NewBookeeping change to Bookeeping with v7r3 (some issue with running jobs when we switch the services. no clear explanation of the problem yet as in principle there is no local caching of the CS in the running job)
lhcbp05.pic.es ready to be put in production
New version for swimming in certification v7r4.
RUN number is set in JDL by DIRAC, in WMS ..TO be checked why ?
Reports from developers:
1 - core : migration to new helper handler is foreseen after v6r2 is released
Properties defined in the CS. We will send our requirements for new properties and will be checked by Adri.
ConfigTemplate : if new info defined in configtemplate we can not use the cLI to install new Agent. Even if we update the LHCbDirac client, the ConfigTemplate are not taken (problem of location) On option is to be able to specify where to look for ConfigTemplate for th extension
2 - WMS
Introduction of job delay for site: for some type we can limit the number of jobs starting at the same time. (scheduling move to operations section)
WMS server EMI : caching of proxy has changed and is causing problem for LHCb. Some solution have been found and will be implemented on our side
3- DMS
pull for new getRepplicas with bulk request.
first prototype for popularity
-add information in dirac-dms-lfn-replicas for option -a: give the LFC status of the file . Without this option, only active replicas are listed. Nice printout provided as well as using the DMScript class for handling switches.
- new functionality in DMScript.py. BK query class not yet split from DMScript.
Documentation written by K.
4 - Resources
GitHub:
-14 issues opened
Savannah:
- 13 issues opened
Summary:
* RSS
- RSS refactoring according to ( https://github.com/DIRACGrid/DIRAC/issues/519 ) - second pull
- SLS LFC sensors in production. Took Stefan, Joel and myself almost a couple of days to understand what was wrong.
Seems middleware is not perfect..
- SLS alarms ( from Stefan ), to be understood how to send them.
- More code cleanup in LHCbDIRAC
* Testing / Certification
- Installing all needed packages from quattor, is simply impossible. Some of them were installed by hand, anothers were
installed using quattor. Servers are installed from the quattor template - unfortunately Tomcat is an incredibly old version, so plots
are not rendered properly.
- Accesible for members of lhcb_dirac on https://volhcb31.cern.ch/jenkins/
- As many of thte packages are not in quattor, running tests is little bit tricky. Now, needed packages were compiled inside
the DIRAC Externals.
- Running nosetests & pyLint ( only on RSS as proof of concept ).
- Some packages in DIRAC and LHCbDIRAC make nose crash, inspecting them one by one.
- Written Javascript script for the pylint page, using it as it is by default turns out to be hard.
5 - production
- fixes in removal plugins, allowing better control of where to remove files
- new method for ranking SEs for data broadcast, using a weight. Still to be determined: where does the weight come from (currently all set equal)? One (static) possibility from CS (OK for archive for example), but for disk SEs, we could use RSS information (using SLS probes) returning the amount of free space. Questions: are SLS probes reliable enough (i.e. what to do when reports 0 free space by mistake, or SLS probes all fail? caching is a solution and requirement on continuity of the value)? are pledges sufficient? how does one relate Dirac SEs to the previous information that is per space token? A method getFreeSpaceForSE(se) provided by any service is OK.
- default for MC broadcast set to one archive only, as for ArchiveDataset plugin.
- Added extensions for WorkflowTaskAgent in LHCbDirac: uses LHCbDIRAC version of TaskManager, that uses LHCbJob (instead of the DIRAC API Job). The RunNumber, when present, is passed by the TransformationTasks to the workflow of the production jobs
- use of ProdConf for setting the application for production
gaudiconfiguration will be generated by ProdConf
6 BKK:
BUGFIX: task 24950: Change step of production.
-the filtering by output file did not worked correctly. When the Output and Input file types are provided then the output file type is removed from the selection.
-The web portal (Step manager) has to be fixed in order to see the result.
Accounting:
NEW: task 16839: Resurrect step level accounting records-report number of events / run number
-new accounting web page is implemented.
-various plots are available in the development system.
7 cert and doc
Jenkins server : integration tools. During the evaluation we raise the issue about coding convention. (Coding convention : in developer guide).
8 web portal
Production request page has been fixed.
AOB :
Machine to be set as DIRAC site : to test new workflow and customization production. (Action joel)
Present : Adri, Federico, Elisa, Vladimir, Mario, Andrei, Matvey, Zoltan, Joel, Ricardo, Krys, Stefan, Antonio
Excused : Philippe
Previous minutes:
DEV and CERT proposal : accepted
DIRAC, DiracWeb :
v6r2 : advanced test for transfer done in Online. So far tests are OK. StorageUsage faster with the new Catalog. Should include a pull to take more than one file catalog in in the API. Release date : pre-release tomorrow and Release next week.
Diracweb : profile for presenter : work in progress
Is it possible to share presenter page ? code is in the pipeline
Jobscheduling should go to v6r3 series.
Other VO : City, SuperB testing of Dirac against Amga. Dirac services installed in Asian region.
2 students working on Boinc and National Chinese Grid.
PIC tested cernvm image on some cloud. (VMDIRAC project)
LHCbDirac, LHCbDiracWeb:
NewBookeeping change to Bookeeping with v7r3 (some issue with running jobs when we switch the services. no clear explanation of the problem yet as in principle there is no local caching of the CS in the running job)
lhcbp05.pic.es ready to be put in production
New version for swimming in certification v7r4.
RUN number is set in JDL by DIRAC, in WMS ..TO be checked why ?
Reports from developers:
1 - core : migration to new helper handler is foreseen after v6r2 is released
Properties defined in the CS. We will send our requirements for new properties and will be checked by Adri.
ConfigTemplate : if new info defined in configtemplate we can not use the cLI to install new Agent. Even if we update the LHCbDirac client, the ConfigTemplate are not taken (problem of location) On option is to be able to specify where to look for ConfigTemplate for th extension
2 - WMS
Introduction of job delay for site: for some type we can limit the number of jobs starting at the same time. (scheduling move to operations section)
WMS server EMI : caching of proxy has changed and is causing problem for LHCb. Some solution have been found and will be implemented on our side
3- DMS
pull for new getRepplicas with bulk request.
first prototype for popularity
-add information in dirac-dms-lfn-replicas for option -a: give the LFC status of the file . Without this option, only active replicas are listed. Nice printout provided as well as using the DMScript class for handling switches.
- new functionality in DMScript.py. BK query class not yet split from DMScript.
Documentation written by K.
4 - Resources
GitHub:
-14 issues opened
Savannah:
- 13 issues opened
Summary:
* RSS
- RSS refactoring according to ( https://github.com/DIRACGrid/DIRAC/issues/519 ) - second pull
- SLS LFC sensors in production. Took Stefan, Joel and myself almost a couple of days to understand what was wrong.
Seems middleware is not perfect..
- SLS alarms ( from Stefan ), to be understood how to send them.
- More code cleanup in LHCbDIRAC
* Testing / Certification
- Installing all needed packages from quattor, is simply impossible. Some of them were installed by hand, anothers were
installed using quattor. Servers are installed from the quattor template - unfortunately Tomcat is an incredibly old version, so plots
are not rendered properly.
- Accesible for members of lhcb_dirac on https://volhcb31.cern.ch/jenkins/
- As many of thte packages are not in quattor, running tests is little bit tricky. Now, needed packages were compiled inside
the DIRAC Externals.
- Running nosetests & pyLint ( only on RSS as proof of concept ).
- Some packages in DIRAC and LHCbDIRAC make nose crash, inspecting them one by one.
- Written Javascript script for the pylint page, using it as it is by default turns out to be hard.
5 - production
- fixes in removal plugins, allowing better control of where to remove files
- new method for ranking SEs for data broadcast, using a weight. Still to be determined: where does the weight come from (currently all set equal)? One (static) possibility from CS (OK for archive for example), but for disk SEs, we could use RSS information (using SLS probes) returning the amount of free space. Questions: are SLS probes reliable enough (i.e. what to do when reports 0 free space by mistake, or SLS probes all fail? caching is a solution and requirement on continuity of the value)? are pledges sufficient? how does one relate Dirac SEs to the previous information that is per space token? A method getFreeSpaceForSE(se) provided by any service is OK.
- default for MC broadcast set to one archive only, as for ArchiveDataset plugin.
- Added extensions for WorkflowTaskAgent in LHCbDirac: uses LHCbDIRAC version of TaskManager, that uses LHCbJob (instead of the DIRAC API Job). The RunNumber, when present, is passed by the TransformationTasks to the workflow of the production jobs
- use of ProdConf for setting the application for production
gaudiconfiguration will be generated by ProdConf
6 BKK:
BUGFIX: task 24950: Change step of production.
-the filtering by output file did not worked correctly. When the Output and Input file types are provided then the output file type is removed from the selection.
-The web portal (Step manager) has to be fixed in order to see the result.
Accounting:
NEW: task 16839: Resurrect step level accounting records-report number of events / run number
-new accounting web page is implemented.
-various plots are available in the development system.
7 cert and doc
Jenkins server : integration tools. During the evaluation we raise the issue about coding convention. (Coding convention : in developer guide).
8 web portal
Production request page has been fixed.
AOB :
Machine to be set as DIRAC site : to test new workflow and customization production. (Action joel)
There are minutes attached to this event.
Show them.