Choose timezone
Your profile timezone:
PASTE meeting
→
Europe/Zurich
Description
CERN, bat 2 R-030
To join the telephone conference, click here: https://audioconf.cern.ch/call/0122882
To dial in to the conference:
a. Dial +41 22 7676000
b. Enter access code 0122882
Production and PASTE Meeting, 15th April 2008, CERN
Chair: A.Tsaregorodtsev
Minutes: S. Paterson
CERN: G.Castellani, Z.Mathe, S.Paterson, A.C.Smith
Phone: R.Bernet, N.Brook, U.Marconi, M.Sapunov, A.Tsaregorodtsev
Apologies: M.Bargiotti, P.Charpentier, J.Closier, R.Graciani, R.Santinelli
0. Ongoing Action List
(01) - 06/11/07 - Gianluca, Marianne, Joel - Remedy the Exceeded Max Staging Retry jobs and update ProcDB via development of Agents to resolve this
(04) - 13/11/07 - Marianne - RAL and PIC migrations, FTS job submissions
(07) - 27/11/07 - Zoltan, Elisa, Marianne - Prototype bookkeeping system on volhcb07
(09) - 04/12/07 - Roberto, Marcos, Gianluca - Develop a general service monitoring framework leading to a virtual control room
(27) - 08/01/08 - Andrew - Backup DMS services on lxgate34 to allow migration to SLC4, port IntegrityDB to DIRAC3
(46) - 11/03/08 - Angelo, Roberto - Send some test jobs to CNAF to test StoRM
(50) - 01/04/08 - Gianluca - Investigate the software installation / availability failures of recent production jobs
(51) - 01/04/08 - Joel - Prepare production tool 'wish-list' in advance of DIRAC3 week
(52) - 01/04/08 - Stuart, Andrei - Plan Production Manager training session
(55) - 08/04/08 - Marianne - Recover data that failed to be registered in the BK DB
(00) - 15/04/08 - DIRAC developers - DIRAC3 (ongoing)
1. Production Status Report
Minutes from the production meeting are available here.
- Remedy Exceeded Max Staging Retry jobs and update ProcDB - Action(01)
Gianluca has checked 65k disk replicas that have been changed to 'a priori good'. The next step is to pass the
information to the IntegrityDB and remove traces of these files. For the DIRAC3 IntegrityDB it was suggested to have
plots showing the accumulation and treatment of files over time to track the activity. Up to now, no tape replicas
have been checked, Gianluca was asked to estimate the number of affected tape replicas remaining in the
ProcessingDB.
- RAL / PIC migrations - Action(04)
Marianne is absent.
- Bookkeeping prototype - Action(07)
Zoltan is now testing the mechanism for view creation. Extra tables are required in the database to facilitate this.
Elisa is working on the migration of the old data to the new schema. Andrei suggested that tests be performed on the
efficiency of the view creation as soon as possible. Zoltan will endeavour to prepare some results for next week's
meeting.
- Monitoring framework - Action(09)
After discussion last week regarding the list of services compiled by Roberto this document will be updated to include
the architecture highlighting the agents and services required to support the activity.
- lxgate34 - Action(27)
Andrew confirmed that the retirement of lxgate34 was pending the transferral of all DM components to DIRAC3. This
is now pending a couple of agents and should be completed in the coming weeks.
- Test jobs to CNAF to test StoRM - Action(46)
Gianluca had nothing to report since the end of the CNAF downtime. Umberto commented that Roberto had made
some transfer tests confirming the several 10s of MB/s required.
- Software Area Corruptions - Action(50)
The problem with LHCb_config.py still exists and Gianluca realised the responsibles were on vacations.
- Production manager training session - Action(51)
A preliminary agenda has been prepared, Stuart will add this to indico after the meeting.
- ~20-30k Problematic IntegrityDB files - Action(54)
It was decided to merge this point with Action(27) and ensure the DIRAC3 integrity machinery can identify and
resolve these cases.
Recent productions were failing after recent CERN downtime, Gianluca had not received acknowledgement that it was
back up. Stuart confirmed that stripping tests had been running yesterday evening with DIRAC3 so this could be
unbanned and test jobs submitted. Some of the stalled jobs at NIKHEF were rescheduled but a 'proxy too short'
warning was received.
2. Production Operations Issues
- Introducing the existing storages/data to DIRAC3 and SRM2 storages checks
DIRAC2 SEs must be reintroduced in DIRAC3 and pointed to the correct space token, Andrew agreed to do this. For
the CCRC phase2, the _USER spaces must also be introduced in the DIRAC3 CS. Andrew has a unit test suite that
checks the SE functionalities and will endeavour to run this regularly to check for any configuration problems.
3. DIRAC3 Tasks
- Stripping tests
Stuart reported ongoing problems with the software installation. Difficult to disentangle the application errors from
the warnings associated to the DaVinci version (official stripping version will only be released on the 28th April).
- Transfers
Andrew agreed to keep a background level of transfers to keep the machinery active. The mechanism to transfer the
stripping outputs should also be tested.
- Accounting
Documentation is being provided for the Accounting system and should be added to the workflow and job wrapper
when available.
- Request Management
Andrei is starting to stuff the workflow with the request machinery.
- Input Data Policy
Stuart conveyed the initial instance of the downloading input data plugin has been made for DIRAC3, this allows
global settings of input data policy and site-specific settings that can override this. In the case where not enough disk
space is available for all the input data requirement plus a sensible buffer, none of the data will be downloaded.
4. AOB
None
There are minutes attached to this event.
Show them.