Production and PASTE Meeting, 27th February 2007, CERN
Chair: A.Tsaregorodtsev
Minutes: S. Paterson
CERN: M.Bargiotti, A.Casajus, G.Castellani, P.Charpentier, J.Closier, R.Graciani, R.Nandakumar, S.Paterson, R.Santinelli, A.Smith, A.Tsaregorodtsev
Phone: M.A.S.Miguelez
1. Production Status Report
Minutes from the Monday meeting are available
here. Joel reported that MC was running smoothly. A new record of
concurrent jobs has been achieved (9715).
Stuart reported that analysis test jobs had successfully run at all Tier-1 sites except RAL with v2r13p3. At CNAF, Joel
reported there was a staging issue causing reconstruction jobs to fail. A meeting was called to debug reconstruction
jobs at the DCache Tier-1 sites on Friday.
Philippe suggested to resurrect the Filling Mode for reconstruction jobs, adding a delay between rescheduling in order
to allow data to be staged. ( ACTION: Stuart, Andrei ) In parallel, the CPU normalisation should be treated
consistently, Stuart suggested following the normalisation used in Gaudi. ( ACTION: Stuart )
It was also mentioned that ProductionEnv should not update the MyProxyServer credential in the case where the
validity is already very large (e.g. to the expiry of the certificate). ( ACTION: Joel )
Pilots sent to LeSC have been continuously failing. A site misconfiguration is suspected and Roberto reported that the
LCG wrapper could not create a scratch directory. Philippe also pointed out that there was no CPU limit, only wall
clock time specified at the site. These issues are to be resolved before reentering LeSC into the mask. ( ACTION:
Roberto, Raja )
2. Production Operations Issues
In order for the gLite Agent Director module to be used in production, the full chain in gLite must be tested. (
ACTION: Gianluca ) Roberto conveyed the gLite equivalent of EDG_WL_JOBID to Stuart and it was agreed to treat this
parameter correctly in the Job Wrapper. ( ACTION: Stuart )
Gianluca reported that the Dashboard developers will provide a service to send information from DIRAC (timescale of a
few weeks).
Andrei announced that fixes are in place for the VO-boxes. Ricardo found that the LFC client is hanging the Transfer
Agents. This is because the Python threading mechanism assigns a slot and if this becomes blocked, all threads hang.
To resolve this it was agreed that all calls to external libraries in DIRAC should be wrapped in a timeout.
With the cause of the hanging Transfer Agent determined, it was agreed to run thousands of threads in order to
stress the system. ( ACTION: Raja, Andrei)
Marianne reported the progress with removing old SE names from the LFC. Unfortunately the LFC binding was not
working which left the replica unchanged when the command was tried. This should be checked with the developers
(has since been resolved). ( ACTION: Marianne )
3. DIRAC3 Tasks
Ricardo reported that the new Configuration Service was in place. There was ongoing work to integrate the new
components with the coding conventions. ( ACTION: Ricardo, Adria )
Andrei confirmed that lxgate14 will become the DIRAC3 testbed. The User system is to be migrated to volhcb01. (
ACTION: Andrei ) LogSE tests are to be performed and Philippe suggested the browser should be moved to LHCb logs.
( ACTION: Joel )
Joel reported that the CVS repository for DIRAC3 was in place. Instructions and package responsibles have been
documented on the
DIRAC Project Page. Work on the build tools is ongoing. ( ACTION: Joel )
4. AOB
Andrew reported that the data deletion was ongoing and that some sites had already responded. ( ACTION: Andrew )
There are minutes attached to this event.
Show them.