PASTE meeting

→ Europe/Zurich
1/1-025 (CERN)

1/1-025

CERN

20
Show room on map
Andrei Tsaregorodtsev
Description
CERN, bat 2 R-030 To join the telephone conference, click here: https://audioconf.cern.ch/call/0165202 To dial in to the conference: a. Dial +41 22 7676000 b. Enter access code 0165202
Production and PASTE Meeting, 3rd June 2008, CERN Chair: A.Tsaregorodtsev Minutes: S. Paterson CERN: A.Casajus, G.Castellani, P.Charpentier, J.Closier, Z.Mathe, S.Paterson, R.Santinelli, M.Sapunov, V.Romanovsky, A.C.Smith, A.Tsaregorodtsev Phone: R.Bernet, N.Brook, R.Graciani, U.Marconi Apologies: 0. Ongoing Action List (01) - 06/11/07 - Gianluca, Marianne, Joel -Remedy input data not available and catalog access issues then update ProcDB (04) - 13/11/07 - Marianne, Andrew - RAL, FTS job submissions (07) - 27/11/07 - Zoltan, Elisa, Marianne - Prototype bookkeeping system on volhcb07 (09) - 04/12/07 - Roberto, Marcos, Gianluca - Develop a general service monitoring framework leading to a virtual control room (27) - 08/01/08 - Andrew - Backup DMS services on lxgate34 to allow migration to SLC4, port IntegrityDB to DIRAC3 (55) - 08/04/08 - Marianne - Recover data that failed to be registered in the BK DB (56) - 29/04/08 - Roland, Stuart, Joel - Provide Production Manager guide (00) - 03/06/08 - DIRAC developers - DIRAC3 (ongoing) Completed Actions Done - (58) - 13/05/08 - Roberto, Joel - Resolve issue with shared area at LCG.Brunel.uk Done - (59) - 13/05/08 - Andrei, Angelo - Check StoRM SE definition in DIRAC2 Done - (60) - 13/05/08 - Andrei - Add LFC synchronization to addDirectory call 1. CCRC Status Nick agreed to take minutes for this session and post them to the LHCb CCRC'08 ELOG. 2. Production Status Report Minutes from the production meeting are available here. - ProcDB update - Action(01) The agent is running but work proceeds in order to remedy 'Input Data Not Available' and 'Catalog Access' failures. - RAL Migration - Action(04) Andrew was very happy to announce there were no tapes remaining to be migrated. There are several files leftover from the first pass to correct and the LFC / CS will be updated accordingly. The timescale to completion is Thursday. - Bookkeeping prototype - Action(07) Zoltan is continuing with Oracle / AMGA tests and expects to have something prepared for the software week in two weeks time. Philippe summarized the outcome of the recent Bookkeeping meeting, Elisa is continuing with the migration of existing data and is implementing the suggestions from the Online group. The migration is complicated by the ongoing DC06 activity. There was some discussion regarding the migration to the new Bookkeeping but it was agreed to postpone any migration plans until the software week. - Monitoring framework - Action(09) Nothing new to report. - Backup DMS services on lxgate34, port IntegrityDB to DIRAC3 - Action(27) Ongoing. Philippe mentioned that integrity agents must be in place in DIRAC3 to allow the recreation of jobs after file replication / removal. Procedures must be put in place to cope with this. For the IntegrityDB, Gianluca mentioned ~21k files are now remaining. - Recover data that failed to be registered in the Bookkeeping - Action(55) Marianne is absent. Andrew agreed to investigate the existing BK xml requests to see if another attempt to add these files will recover any of the discrepancy. - Production Manager Guide - Action(56) Roland is on shift next week so will work on this then. - Add LFC synchronization to addDirectory call - Action(60) Philippe confirmed that the LFC synchronization is working but mentioned that the restart of transformations was not enough to ensure a new mask was taken. As an aside, the file mask in DIRAC2 is limited to 256 characters, Andrei agreed to check the status of this in DIRAC3. 3. Production Operations Issues - DIRAC2 Production issues Philippe remarked that it is impossible to run at IN2P3, the list-match reports 0 available CEs and this will be investigated. The failover of log files can create inconsistent behaviour since in this case a tarfile is present instead of an unpacked directory. This type of jobs should not be rescheduled. Shifters for DIRAC2 should not submit any jobs until the recent patches are tested and a new release is made. There is pressure from the sites to retire SRMv1. - Aborted pilots in DIRAC2 Ricardo reported that all pilots are being aborted with 'maradonna' errors for the first few hundred jobs in the queue. This issue will be investigated with some urgency by the developers. 4. DIRAC3 Tasks A summary of the issues to be resolved during the DIRAC3 week are: - Next version of the WF modules (failover,accounting, ProcDB reporting, etc) - Independent WF library distribution - MC, user analysis workflows, jobs - Setting up test system on volhcb01 - Components profiling, optimization - LFC Replica status treatment Ricardo raised the issue of python2.4 / python2.5 and how we can obtain python bindings for python2.5 in the application area, Philippe agreed to check up on this point. 5. AOB - Next Production Shift manager tutorial Andrei and Stuart suggested that the production shift manager tutorial proposed for the software week should be postponed such that efforts can be redirected to DIRAC3 commissioning. Philippe mentioned that the IN2P3 VO-box is still not working after the recent update from Raja. Andrei agreed to quickly investigate this problem.
There are minutes attached to this event. Show them.
    • 09:30 → 09:50
      CCRC operations issues 20m
    • 09:50 → 10:10
      Ongoing actions 20m
      (01) - 06/11/07 - Gianluca, Marianne, Joel - Remedy the Exceeded Max Staging Retry jobs and update ProcDB via development of Agents to resolve this (04) - 13/11/07 - Marianne, Andrew - RAL, FTS job submissions (07) - 27/11/07 - Zoltan, Elisa, Marianne - Prototype bookkeeping system on volhcb07 (09) - 04/12/07 - Roberto, Marcos, Gianluca - Develop a general service monitoring framework leading to a virtual control room (27) - 08/01/08 - Andrew - Backup DMS services on lxgate34 to allow migration to SLC4, port IntegrityDB to DIRAC3 (55) - 08/04/08 - Marianne - Recover data that failed to be registered in the BK DB (56) - 29/04/08 - Roland, Stuart, Joel - Provide Production Manager guide (58) - 13/05/08 - Roberto, Joel - Resolve issue with shared area at LCG.Brunel.uk (59) - 13/05/08 - Andrei, Angelo - Check StoRM SE definition in DIRAC2 (60) - 13/05/08 - Andrei - Add LFC synchronization to addDirectory call (00) - 27/05/08 - DIRAC developers - DIRAC3 (ongoing)
    • 10:10 → 10:25
      Production status report 15m
      - Report from the Production meeting:
    • 10:25 → 10:55
      Production operations issues 30m
      - Aborted pilots in DIRAC2
    • 10:55 → 11:25
      DIRAC3 tasks 30m
      DIRAC3 week main topics (preparing DIRAC3 for production) - Next version of the WF modules (failover,accounting, ProcDB reporting, etc) - Independent WF library distribution - MC, user analysis workflows, jobs - Setting up test system on volhcb01 - Components profiling, optimization - LFC Replica status treatment
    • 11:25 → 11:35
      AOB 10m
      - Next Production Shift manager tutorial