Production and PASTE Meeting, 27th March 2007, CERN
Chair: A.Tsaregorodtsev
Minutes: S. Paterson
CERN: M.Bargiotti, G.Castellani, P.Charpentier, J.Closier, A.Smith, S.Paterson, R.Santinelli
Phone: R.Bernet, J.Blouw, N.Brook, C.Cioffi, R.Graciani, U.Marconi, R.Nandakumar, A.Tsaregorodtsev
1. Production Status Report ( J.Closier )
Minutes from the Monday meeting are available
here.
The Reconstruction activity is pending the successful tests of the Stager. The Castor 1 data access issues at PIC are
being investigated. Stuart reported that the files from test jobs to CNAF are yet to be staged. Angelo reported at the
production meeting that these files were problematic and further tests should be performed.
Andrei reported that the files Marianne requested to be cleaned up have now been removed at IN2P3. These files
should now be removed from the LFC, Philippe mentioned that the SIM and DIGI for the same productions can be
cleaned as well ( ACTION: Marianne ).
2. Production Operations Issues
Stuart reported that the Stager will be installed on the Production system this afternoon and that the Data Optimizer
has been updated with the functionality to request ancestor files using the genCatalog utility. Reconstruction and
Stripping jobs are to be submitted for testing. ( ACTION: Stuart, Joel).
Further to the issue of bad quality data being reconstructed (phys-lumi2 and phys-lumi5) Marianne has a list of files
from Carmine and will proceed to remove them from the LFC. A list of DST replicas at CERN is being compiled by
Marianne who expects the results by the end of the day. ( ACTION: Marianne )
Roberto performed tests using lcg-clean to determine the speed of removal. Using 9000 files at RAL (each of 15KB), it
took over 8 hours using 20 threads for the removal. This triggered discussion on how best to proceed with large scale
file removals. Roberto reported that Tier-1 site administrators were expecting removal requests and without official
tools it was agreed to continue with this approach.
Following investigation, Ricardo found that some transfers to CERN-disk failed since replicas already existed on
CERN-tape (the opposite also occurred to a lesser extent). Due to the failover mechanism of the Transfer Agent,
other SEs can transfer files to CERN-tape. A fix should be put in place for the Transfer Agent logic ( ACTION: Andrei ).
After some discussion about pending transfer requests it was agreed to follow a suggestion from Andrew. A list of all
DST files at CERN should be made, then the remainder can be transferred to CERN and all pending DST requests can
then be removed from the VO-boxes. This will reduce the problem of available disk space at the Tier-1 sites and allows
us to move all data to the Tier-1s from scratch. The timescale for this operation should be compiled. ( ACTION:
Andrew )
In order to fully switch to VOMS proxies and standardise on VOMS proxies in the WMS, the Job Receiver must be
equipped with a check on the submitted proxy and automatically extend the proxy when required. ( ACTION:
Gianluca) Roberto confirmed that the proxy extension script from E.Lanciotti works without myProxyServer
credentials. Gianluca commented that WMproxy submission fails if the role changes. Stuart suggested that a simple
and effective policy for LHCb would be to map all iusers to /lhcb by default and use that role throughout. For
additional roles, the CS mappings based on DIRAC roles can be used. Roberto confirmed that the current situation is
that there are three groups for LHCb (user, production, sgm) which is sufficient for this approach.
For the issue of LFC files registered without replicas, Stuart reported that a very preliminary investigation revealed no
problems in the logic of copying and registering the files. It was agreed for Andrew to proceed with this task. (
ACTION: Andrew )
Andrei reported on the User system (lxgate05) migration to volhcb01. The port scan requests have been made but
not yet satisfied. Joel agreed to chase this up. ( ACTION: Joel ) The VO-boxes had a CS connection problem since
ports 9130 and 9131 were closed. Andrei confirmed that these ports were specified in the original document and this
should be investigated. ( ACTION: Raja )
3. DIRAC3 Tasks and Databases Discussion
Ricardo has updated the database schema and reports a few changes for DIRAC3. By default, a fixed rather than
infinite length queue will be initiated that will limit the number of connections to the DB. This allows a steady number
of open connections to the DB which can be extended if requested and in this way idle connections do not force a
server restart. Other modifications include increased error reporting.
Andrei uploaded a proposal for the DIRAC3 database structure (
document) which was thoroughly discussed. A
summary of the main points includes renaming of the Job Logging DB and adding the Stager DB (distinct to the Job
Database) to the list. Andrei agreed to revise the document for a forthcoming meeting. ( ACTION: Andrei )
4. CHEP07 Abstracts
Almost all of the abstracts were updated to the agenda page as requested by Philippe. A meeting to discuss the
abstracts was scheduled for Thursday at 3pm. For the joint Ganga-DIRAC abstract, Stuart was requested to trigger
discussions with Andrew in advance of the meeting. ( ACTION: Stuart )
5. AOB
Joel mentioned a problem with the ProcessingDB had been resolved by Gennady but requested that the recipe for
solving this should be published.
There are minutes attached to this event.
Show them.