Choose timezone
Your profile timezone:
PASTE meeting
→
Europe/Zurich
Description
CERN, bat 2 R-030
To join the telephone conference, click here: https://audioconf.cern.ch/call/0122882
To dial in to the conference:
a. Dial +41 22 7676000
b. Enter access code 0122882
Production and PASTE Meeting, 18th December 2007, CERN
Chair: A.Tsaregorodtsev
Minutes: S. Paterson
CERN: G.Castellani, P.Charpentier, J.Closier, S.Paterson, R.Santinelli
Phone: R.Bernet, N.Brook, G.Kuznetszov, U.Marconi, M.A.S.Miguelez, M.Sapunov, A.C.Smith, A.Tsaregorodtsev
Apologies: M.Bargiotti, R.Nandakumar
0. Ongoing Action List
(01) - 06/11/07 - Gianluca, Marianne - Remedy the Exceeded Max Staging Retry jobs and update ProcDB (pending resolution of each case)
(02) - 06/11/07 - Gianluca, Marianne - Agents for automatic resolution of IntegrityDB entries should be developed
(03) - 30/10/07 - Marianne - Update of LFC replica flags (pending NIKHEF)
(04) - 13/11/07 - Marianne - RAL and PIC migrations, FTS job submissions
(05) - 27/11/07 - Roberto - Oversee LFC intervention plan for upgrade to latest version (testing)
(06) - 27/11/07 - Andrei - Prepare draft pilot document as internal LHCb note
(07) - 27/11/07 - Elisa, Marianne - Prototype bookkeeping with API wrapped in DISET service (volhcb07 will be used for this)
(08) - 04/12/07 - Alexei, Andrei - Further investigate cause of sporadic failures at GridKa for production jobs (pending tests with v2r18p5)
(09) - 04/12/07 - Gianluca, Roberto - Prepare an SRM sensor test for the LHCb SAM framework (pending upload to SAM DB)
(11) - 11/12/07 - Marianne - Clean up SIM/DIGI for production IDs 1957, 1960, 1962
(14) - 11/12/07 - Joel - Update lhcb-proxy-init with small wrapper to ensure environment is not changed
(16) - 11/12/07 - Joel - Trigger machine upgrades (volhcb06, port 80 not yet visible outside CERN)
(17) - 11/12/07 - Joel - Prepare MDF and signal processing workflows (test with v2r18p4 / v2r18p5)
(18) - 18/12/07 - Gianluca - Check current status of SAM jobs on Test system
(19) - 18/12/07 - Roberto,Joel - Prepare draft of the LFC Admin service interface
(20) - 18/12/07 - Roberto,Andrei - Investigate recent failures of uploading logs (volhcb06) check VO-boxes for Transfer Agent problem
(21) - 18/12/07 - Roberto,Gianluca - Test RFIO at CNAF, coordinate with local site contacts and Castor support
(22) - 18/12/07 - Marcos - 3 new requested productions should be created and submitted, 1911 should be extended
(00) - 18/12/07 - DIRAC developers - DIRAC3 (ongoing)
Completed Actions
Done - (10) - 04/12/07 - Gianluca - Report problematic CERN tapes to the Castor support team - List for last month sent yesterday, now an everyday activity
Done - (12) - 11/12/07 - Gianluca - Check User / Production masks and bring up to date - Some sites added
Done - (13) - 11/12/07 - Matvey - Add a flag to lhcb-proxy-init for the length of the proxy - Implemented
Done - (15) - 11/12/07 - Roberto - Investigate missing myproxy-* commands in 3.1 UI -Now resolved
1. Production Status Report
Minutes from the production meeting are available here.
Gianluca reported problems with the SAM tests on the Test system, Andrei requested that the current status should
be checked ( ACTION: Gianluca ). Stuart mentioned that migration to the Production system could be made but the
effort would be better placed in DIRAC3.
2. Production Operations Issues
- LFC upgrade - Action(05)
All information is summarized on a webpage circulated by Roberto, including hourly test results. Currently 5 instances
are up and running but flickering behaviour is observed at RAL (Jean-Philippe and Lana were triggered to investigate).
LFC administrative commands are to be checked. Joel reported there was no way to list which DNs are registered for
any particular VO in the LFC. Andrei requested a draft of the service interface for LFC admin, any missing functionality
should be requested from the developers. ( ACTION: Joel,Roberto ) RAL and IN2P3 can be added to the LFC mirrors in
DIRAC2.
- v2r18p3 release - Actions (08) and (17) - New workflows and testing with current fixes for GridKa
Philippe mentioned the lack of progress for productions before Christmas and how to remedy this. Problems with the
MDF production and reprocessing of the signal remains to be solved. It was suggested to revert the Gaudi
Application module to the last stable version (except for GridKa) and create a new tag (v2r18p5) for development.
Joel agreed to check these workflows with this distribution v2r18p4( ACTION: Joel ) Andrei will continue testing at
GridKa after compiling v2r18p5. ( ACTION: Andrei )
- Log SE at volhcb06 - Action (16)
Port 80 not yet visible outside CERN. Recent failures in the uploading of logs on volhcb06 should be investigated and
Roberto agreed to check this server-side. ( ACTION: Roberto )
- User proxy problems
Stuart confirmed that jobs were running again on the user system after the latest instructions from Andrei to the
users. After Action(14) is complete, can better coordinate approach with Ganga team.
- Use of classical SE
A classical SE is being used for log files but will eventually be phased out by GD. For a private SE, this is fine and for
DIRAC3 may be switched to a DISET SE.
- RFIO at CNAF
The recent data access problems at CNAF have triggered discussion about the installation of a rootd server. Tests with
opening a file locally at CNAF should confirm the status of RFIO before proceeding. Roberto has access to CNAF and
agreed to make the RFIO test. ( ACTION: Roberto )
3. DIRAC3
Stuart conveyed that the full Gaudi application chain and a VO-specific DIRAC API for LHCb jobs is now working. One
of the recent challenges in DIRAC3 was how to provide VO-specific plugin modules and a mechanism is now in place
for software distribution. This generic approach to plug-in modules will also be extended to input data treatment.
Problems have been encountered with the available distribution (GFAL/LFC libraries). Different behaviour for different
UI versions has also been observed. Joel and Stuart will sit together to resolve these configuration issues.
Joel reported that Gaudi Application is now working and the other necessary modules are in the pipeline. Gennady
commented that except for the submission itself, the production service chain is in place. Some debugging of the
submission command on the server side is still required.
Andrei mentioned that the job monitoring page is becoming more functional and all comments are welcome to help
Matvey continue improving this.
4. AOB
During the Christmas period, services will be available on a best-effort basis. Three new signals requested by Olivier
should be submitted before the Christmas break and production 1911 should be extended ( ACTION: Marcos ). The
next PASTE meeting will occur on the 8th January. Philippe suggested an extended Operations meeting for this
Thursday.
There are minutes attached to this event.
Show them.