Choose timezone
Your profile timezone:
PASTE meeting
→
Europe/Zurich
2-R-014 (CERN)
2-R-014
CERN
Description
CERN, bat 2 R-030
To join the telephone conference, click here: https://audioconf.cern.ch/call/0165202
To dial in to the conference:
a. Dial +41 22 7676000
b. Enter access code 0165202
Production and PASTE Meeting, 16th December 2008, CERN
Chair: A.Tsaregorodtsev
Minutes: S. Paterson
CERN: P.Charpentier, J.Closier, Z.Mathe, M.A.S.Miguelez, S.Paterson, R.Santinelli
Phone: R.Graciani, M.Sapunov, A.Tsaregorodtsev
Apologies: A.C.Smith
0. Ongoing Action List
(07) - 27/11/07 - Zoltan, Elisa - Ongoing new bookkeeping system development. Migration plan
(57) - 24/06/08 - Vladimir,Roberto - Automatic publishing of BDII / GOCDB information in the CS using GOCDB site names and provide a summary of recent changes
(83) - 01/09/08 - Stuart,Roberto - To perform glexec tests (pending SCAS certification)
(90) - 09/09/08 - Joel,Stuart - Use new ProductionOutputData JDL parameter in modules instead of creating LFNs
(93) - 16/09/08 - Andrew - Improve RAW data monitoring tools and status of files in online runDB
(94) - 23/09/08 - Andrew, Marcos - Using SystemLogging service for dynamic services status monitoring (Andrew to provide FTS example)
(95) - 23/09/08 - Roberto - SLS and SAM sensors development, upgrade SE tests with new DMS unit test
(96) - 23/09/08 - Roberto - Update Dashboard SAM tests visualization
(106) - 30/09/08 - Ricardo - Produce twiki documentation for dirac-install
(107) - 30/09/08 - Ricardo,Philippe,Joel,Hubert,MarcoC - Produce proposal for how to perform standardised installation using install_project/CMT
(109) - 14/09/08 - Matvey,Andrew - Expose DataLogging interface on the web page
(111) - 14/09/08 - Philippe,Alexei,Andrei - Productions with file merging and associated data distribution implementation
(113) - 21/10/08 - Ricardo, Adri - Document the CS procedures for both the mechanism to define authorization rules in DIRAC and also the current group properties
(115) - 21/10/08 - Vladimir,Matvey - Update SAM Agent to monitor and manage submitted jobs and publish results for faulty CEs to the SAM DB
(116) - 18/11/08 - Ricardo - Circulate a report on the recent Gauss CPU studies, steer the CPU related activities in DIRAC
(117) - 18/11/08 - Ricardo - Add a site attribute to reflect whether generic pilots are accepted there, update the Director
(119) - 16/12/08 - Stuart - Recreate dummy production with updated workflow lib
(120) - 16/12/08 - Ricardo - Update TQ Director to deal with a mixture of generic / private pilots at sites
(121) - 16/12/08 - Andrei, Alexey - Add production extensions to logging information
Completed Actions:
Done - (118) - 25/11/08 - Roberto,Vladimir - Provide script to demonstrate GFAL thread safety issue and submit a Savannah bug
1. Action Updates
- Action(07) - Prototype Bookkeeping
The GUI interface and CLI will deal with a separated config name and version. Joel mentioned that the BK reports will
no longer be sent to the old BK and Zoltan suggested stopping the old BK receiver. Stuart will recreate the dummy
production with this workflow lib version. ( ACTION: Stuart ) Andrei asked about the status of the interface for
querying for datasets from the BK, Zoltan will provide a description of the possible selection criteria. Whether or not
the query can be built from concatenated queries was discussed, Philippe commented that we should only have one
single query per production.
- Action(57) - Automatic publishing of BDII / GOCDB information in the CS
Roberto mentioned that the document was attached to the last meeting agenda with some action points.
- Action(83) - gLexec tests
Nothing new to report, SCAS is still under certification. Deployment of the pilot role at the sites is a slow process.
There was some discussion regarding how to proceed with generic pilots and the conclusion was to go forward with
this policy at the Tier-1s accepting the pilot role. There was some discussion about to proceed with a mixture of sites
accepting generic and private pilots and it was agreed to use the published VO software tag to allow Ricardo to
proceed. ( ACTION: Ricardo )
- Action(90) - Production Output Data parameter
Ongoing, will try to make progress this afternoon.
- Action(93),(94) - Improve RAW data monitoring tools and status of files in online runDB
Ongoing.
- Action(95),(96) - Service Monitoring
LHCb Dashboard has been updated to better display the SAM test results e.g. using the DIRAC naming convention.
The NL-T1 (NIKHEF + SARA) naming issues have finally been resolved. The tool used for accounting site availability
does not take this into account however (GridView). Historical views of data in the Dashboard have also been fixed.
The migration of the SAM tests to the updated client versions has occurred, this allows to submit to the gLiteWMS.
Unfortunately this broke the CE tests temporarily due to a technical issue with AFS, Stuart suggested potentially
migrating this activity to submission via DIRAC, can discuss offline. The SRMv2 SE storage sensors are now
considered as critical, central and local Tier-1 LFC tests are also in place.
- Action(106) - Ricardo - Produce twiki documentation for dirac-install
Ongoing, being prepared.
- Action(107) - Produce proposal for how to perform standardised installation using install_project/CMT
Philippe summarized that the work for this is ongoing, Hubert made a lot of progress with this in instal_project
yesterday. A CMT interface for using dirac-install is under preparation. The package structure of the LCG data
management components has been updated, after some discussion it became clear that there are several outstanding
issues relating to this (namely 1.10.17 of GFAL is unusable). Roberto commented that the stress testing of the
1.10.18 GFAL version (in the preview system) seemed to indicate the thread safety issue is resolved, this version
should be made available this week.
- Action(109) - Expose DataLogging interface on the web page
Ongoing.
- Action(111) - File merging productions
Ongoing, currently Alexey is concentrating on the request form. Philippe mentioned that due to the streaming in the
Stripping jobs we will have to merge each of the streams independently. This would suggest several merging
productions per stripping production. Philippe agreed to summarize discussions regarding file sizes, distribution of
files based on runs etc. on the merging twiki page.
- Action(113) - CS Documentation
Ongoing.
- Action(115) - SAM Agent
Vladimir continues to work on client-side scripts as a first step to automating this action.
- Action(116) - CPU studies
In order to deploy the filling mode we still require a benchmarking factor for the local CPU.
- Action(117) - Generic pilot site attribute
See report in Action (107). Added Vladimir to this action point.
2. Production Status Report
- Report from the Production meeting & FEST preparation
Stuart mentioned that the following issues are most urgent for the FEST:
- BK query population of datasets
- Finalize derived production mechanism
- Optimizations for the creation of jobs (for productions with input data)
Recent high volume production extensions without ELOG entries highlighted a lack of information in the production
logging ( ACTION: Andrei, Alexey ).
Joel mentioned that part of the recent Gauss validation requests will not be submitted due to a bug found in Brunel,
Marco is looking at this.
In order to more easily monitor large numbers of productions in the monitoring page it was suggested to introduce a
ProductionGroup parameter (analogous to JobGroup).
- v4r3p1,2 releases
Andrei mentioned two bug fix releases made for Interfaces, the BK client and Workload Management.
- Plans for Xmas shutdown (if any)
See Production Operations minutes.
3. DIRAC3 Tasks
All points on the agenda discussed above. Joel raised the issue of test jobs sent to an SLC5 CE at CERN and Ricardo
will assist in the debugging.
Stuart mentioned that we should have admin tools to be able to submit (and list-match) pilots locally based on a
given job object. Visualization of Task Queues on the monitoring portal would also be advantageous e.g. TQ
requirements, summary of pilots, number of jobs etc.
4. AOB
The next Paste meeting will be on the 6th January, merry Christmas and happy Hogmanay to all!
There are minutes attached to this event.
Show them.