Production and PASTE Meeting, 12th February 2008, CERN
Chair: A.Tsaregorodtsev
Minutes: S. Paterson
CERN: M.Bargiotti, N.Brook, A.Casajus, G.Castellani, P.Charpentier, G.Kuznetszov, Z.Mathe, S.Paterson, R.Santinelli
Phone: M.A.S.Miguelez, M.Sapunov, A.Tsaregorodtsev
Apologies: J.Closier, A.C.Smith
0. Ongoing Action List
(01) - 06/11/07 - Gianluca - Remedy the Exceeded Max Staging Retry jobs and update ProcDB via development of Agents to resolve this (merged with Action(02))
(03) - 30/10/07 - Marianne - Update of LFC replica flags (pending NIKHEF)
(04) - 13/11/07 - Marianne - RAL and PIC migrations, FTS job submissions
(07) - 27/11/07 - Elisa, Marianne - Prototype bookkeeping with API wrapped in DISET service after ensuring AMGA and Oracle availability on volhcb07
(09) - 04/12/07 - Roberto, Marcos, Gianluca - Develop LHCb SAM framework and summarize architecture for SE tests
(25) - 08/01/08 - Marianne - Resolve why number of processed events is missing in Bookkeeping (also cause of missing files)
(27) - 08/01/08 - Andrew - Backup DMS services on lxgate34 to allow migration to SLC4
(32) - 22/01/08 - Gianluca, Marianne - Identify and resolve faulty productions with duplicated events
(35) - 12/02/08 - Zoltan, Andrei - Setup DIRAC3 BK interface on volhcb05 (install DIRAC3 there)
(36) - 12/02/08 - Roberto, Gianluca - Investigate other experiments usage and functionality of the Dashboard
(37) - 12/02/08 - Andrei - Update proxy handling infrastructure in DIRAC2
(00) - 12/02/08 - DIRAC developers - DIRAC3 (ongoing)
Completed Actions
Done - (30) - 15/01/08 - Gianluca - Test NIKHEF after reinstallation of the shared area
Done - (33) - 05/02/08 - Joel - Check new ProcessingDB functionality and report any problems
Done - (34) - 05/02/08 - Joel - Trigger set up of disk partitions on volhcb01
1. Production Status Report
Minutes from the production meeting are available
here.
- Remedy Exceeded Max Staging Retry jobs and update ProcDB - Action(01)
Gianluca's agent is on the production system this starts from the JobDB and works to mark the replicas problematic in
the ProcessingDB. Whether or not the job should be deleted after this agent has run was raised by Philippe. After
some discussion it was agreed to proceed by deleting these jobs. The agents to work from the Integrity DB and
perform corrective actions is pending.
2. Production Operations Issues
- Update LFC replica flags - Action(03)
Ron conveyed the list of SURLs to Marianne and the task of updating the LFC should be completed by the end of the
week.
- RAL / PIC migration - Action(04)
RAL is ongoing, Marianne is waiting for the next tape to be ready to submit more FTS jobs. PIC catalogue entries are
being updated in the LFC.
- Bookkeeping receiver prototype
Zoltan mentioned that the DIRAC3 service to interface to the existing BK production instance is in place and working.
The DIRAC3 service should be added to volhcb05 to facilitate this (DIRAC3 will have to be installed). ( ACTION:
Zoltan, Andrei )
- Test NIKHEF shared area - Action(30)
The shared area at NIKHEF was cleared by Jeff last week and now jobs are running fine.
- Productions with duplicated events - Action(32)
Gianluca was wondering how to work out the output data file names for a given job. Stuart mentioned that a recipe
for this was already in place in the JobSanityAgent.
After some discussion relating to the Dashboard it was suggested to investigate how the other experiments use it and
what the functionality of the system is. ( ACTION: Roberto, Gianluca )
- IN2P3 shared area
The IN2P3 shared area is currently full, Andrei is waiting for a response from the site admins.
The cause of new event types not appearing in the Boookkeeping has in principle been solved by Zoltan.
Andrei confirmed that the proxy handling infrastructure in DIRAC2 will be updated this afternoon. ( ACTION: Andrei )
3. DIRAC3 Tasks
- Production service on volhcb01
DIRAC3 has been installed on the production machine (volhcb01) by Ricardo. This should still be configured and the
DB schema also needs to be applied. The master CS service should be moved to this machine and a mirror setup on
volhcb03.
- Bookkeeping receiver service - see Action(35)
- Workload Management System
Stuart conveyed that the WMS services are in good shape, MDF tests at CERN and NIKHEF have been successful and
other SRM2 related tests are ongoing.
- Workflow Modules
Joel will continue to progress with the modules this afternoon BK and job finalization are pending.
- Production Management status
Gennady mentioned that the infrastructure to submit jobs is in place. A few hours to implement naming conventions
etc. will be required. There was some discussion of whether giving a meaningful name to the production and using a
production ID.
- Accounting status
Adri is making progress with the Accounting but the tools for visualization are still under preparation. The views must
be defined to allow us to present the information correctly. Adri was asked to send the list of keys for the Transfer
Accounting in DIRAC2 to the developers list as a precursor to defining the schema of the Accounting DB in DIRAC3.
After some discussion it was suggested to use the DIRAC2 Transfer Accounting for CCRC08 in the short term by
writing an agent to extract the information from the TransferDB.
Access to PIC machines from the gateway machine at the pit is to be clarified.
4. AOB
None
There are minutes attached to this event.
Show them.