Choose timezone
Your profile timezone:
PASTE meeting
→
Europe/Zurich
2-R-014 (CERN)
2-R-014
CERN
Description
CERN, bat 2 R-014
To join the telephone conference, click here: https://audioconf.cern.ch/call/0122929
To dial in to the conference:
a. Dial +41 22 7676000
b. Enter access code 0165202
Production and PASTE Meeting, 18th August 2009, CERN
Chair: S.Paterson
Minutes: S.Paterson
CERN: P.Charpentier, Z.Mathe, S.Paterson, V.Romanovsky, A.C.Smith
Phone: R.Graciani, E.Lanciotti
Apologies: J.Closier, M.Rashid, R.Santinelli, F.Stagni, M.Sapunov, A.Tsaregorodtsev, A.Zhelezov
0. Ongoing Action List
(94) - 23/09/08 - Andrew,Marcos - Using SystemLogging service for dynamic services status monitoring (Andrew to provide FTS example)
(116) - 18/11/08 - Ricardo - steer the CPU related activities in DIRAC, document procedures, move to the HEP CPU units
(122) - 13/01/09 - Ricardo,Vladimir - Develop specialized SAM Agent to track and report errors with regard to pilots
(133) - 03/03/09 - Matvey - Provide request DB interface page
(139) - 31/03/09 - Roberto,Stuart - Track and test glexec as it is deployed at the sites, put CE into production pending the official release
(140) - 31/03/09 - Andrew - Formulation of procedure for how to treat banning of sites based on job requirements e.g. input data / staging...
(146) - 21/04/09 - Vladimir,Joel - SAM SW installation logic by synchronizing shared areas with the CS SW specification. Should be complemented with the installed SW monitoring tools.
(151) - 26/05/09 - Andrew - Update central components with optimized BK methods using file time stamps
(162) - 07/07/09 - Ricardo,Andrei - document the TQ priorities algorithm
(163) - 07/07/09 - Joel - investigate possible use of CERN backup tools for log files archiving.
(164) - 16/07/09 - Matvey,Adri - Formalize and simplify the release procedure for new web portal pages (including regular DIRAC version updates)
(165) - 16/07/09 - Ricardo - Add WMS accounting for visualization of 'time to running' and other accounting metrics
(166) - 16/07/09 - Ricardo - Set up volhcb05 as the new certification machine
(167) - 16/07/09 - Philippe,Ricardo,Roberto - SLC5 commissioning & migration
(168) - 04/08/09 - Matvey,Alexey - Release procedure for the web portal, ensure request page is in production
(169) - 04/08/09 - Stuart,Vladimir - Continue SLC5 testing at the sites where this is available
Completed Actions:
Done - (161) - 07/07/09 - Alexei - define Production Request for ongoing production to allow its monitoring
1. Action Updates
- Action(94) - SystemLoggingDB example
Ongoing, Marcos will make a tool available in CVS for this.
- Action(111) - File merging productions & request page
Ongoing.
- Action(116) - CPU studies
Ongoing.
- Action(122) - Develop specialized SAM Agent to track and report errors with regard to pilots
Ongoing.
- Action(133) - RequestDB monitoring
Matvey will circulate the link to the request summary page on lhcbtest for comments. Andrei will make a proposal on
the restructuring of the portal menus.
- Action(139) - gLexec testing after deployment at new sites
Pending the announcement that glexec is deployed at the sites.
- Action(140) - Banning faulty storage elements
Ongoing, technically completed but Andrew will add scripts to streamline this process and is starting on the documentation.
- Action(146) - SAM software distribution synchronization with the CS
Joel prepared a module for listing the software area contents but Vladimir noted that this requires write access to the
shared area. Ongoing.
- Action(151) - Update of central BK components with optimized methods
Ongoing.
- Action(162) - Document the TQ priorities algorithm
Ongoing.
- Action(163) - Investigate possible use of CERN backup tools for log files archiving.
Ongoing.
- Action(164) - Web portal release procedure
Ongoing.
-Action(165) - Add WMS accounting for visualization of 'time to running' and other accounting metrics
Ongoing.
-Action(166) - Set up volhcb05 as the new certification machine
volhcb05 is up and running but Ricardo wasn't sure that the port scan was completed. Philippe confirmed that this
was done and will forward the mail to Ricardo.
-Action(167) - SLC5 commissioning & migration
Philippe summarised that a list of compatible versions of GFAL, lcgutils and VOMS now exists. A new version of the
ldap library seemed to fix the space separator problem. A tarball of the externals was built by Ricardo and is working.
Hubert is preparing an LHCbGrid v2r1 to take these changes into account. DIRAC will now depend on v2r*. The
commissioning of the service installation is over, no problems observed. Reformulated action.
-Action(168) - Release procedure for the web portal, ensure request page is in production
Ongoing.
-Action(169) - Continue SLC5 testing
Most sites don't have a problem except SLC5 IN2P3 where ~50% of jobs consistently failed. This action will be kept
open for debugging this site and checking RAL is ok (in downtime during these tests).
2. Production Status Report
- Report from the Production meeting
Due to the downtime of NIKHEF/SARA some production requests are not yet completed, the MC productions have
been extended in order to complete the requests by brute force. A new production status "ValidatingOutputs" has
been established in order to perform final checks on the output data files for a given production. Still pending
clarification from the CaloCalibrationWG after amendment of the request by the PPG (also a new AppConfig options file
will be required).
- Set up stripping workflow(s)
o starting from RDST
o with reconstruction as first step (starting from RAW or DST for MC)
o how to handle multiple streams as output of the last DaVinci?
There is a request to strip the minimum bias data sample. The stripping should be commissioned during FEST. When
the first-pass reconstruction occurs the possibility of running the stripping in addition was mentioned. This could be a
useful exercise. The stripping preselections will be discussed during meetings in the coming weeks. Whether no
events are available (and the file summary record reflects this) is still to be demonstrated.
- Processing real data by runs (i.e. not mix runs at any stage)
The BKReport module should be checked to ensure the correct propagation of the run numbers.
- Use file size (or number of events) rather than number of files for creating jobs
The code for this exists and will be integrated to the production system in due course.
- Disk quota implementation
Andy reported that there is a web page and command line tool to explore the resource usage.
- User data policy
After some discussion it was agreed to come up with a proposal for user data and discuss with all relevant parties.
3. DIRAC Tasks
- Framework status
The CS merging issue is now understood and resolved, this is not yet put back into production.
- Bookkeeping status
Zoltan added the possibility to see log files to a given date and an ancestor retrieval method similar to the already
existing descendents.
- WMS status
Ricardo mentioned that the DIRAC site development is advancing. The automatic updating of CRLs is now in place.
The TorqueCE configuration is being adapted to support multiple queues.
4. AOB
Philippe circulated an email regarding data cleanup and invited any comments. There was some discussion regarding
productions that were skipped during the migration to the new BK, this cleaning will be made from the LFC /
storages.
Stuart mentioned that a trial for using Savannah is ongoing with the production operations task list, in order to
streamline the PASTE actions list this will be formatted in the same way and circulated before the next meeting. A
'ToBeDiscussed' and 'Ongoing' status will be possible such that only actions to be discussed during the meeting will be
discussed. In case some actions fall by the wayside they will automatically be updated to 'ToBeDiscussed'. Therefore
the action assignees should take care to update this status and a reminder will be sent before the next meeting.
Philippe also requested feedback on the EGI support document sent to the developers list. The most important
meetings for this will be in September and we should take care to ensure the LHCb request is clearly defined before
then.
The next PASTE meeting during the vacations period will be held on 1st September.
There are minutes attached to this event.
Show them.