Choose timezone
Your profile timezone:
PASTE meeting
→
Europe/Zurich
Description
CERN, bat 2 R-030
To join the telephone conference, click here: https://audioconf.cern.ch/call/0122882
To dial in to the conference:
a. Dial +41 22 7676000
b. Enter access code 0122882
Production and PASTE Meeting, 3rd July 2007, CERN
Chair: P.Charpentier
Minutes: S. Paterson
CERN: M.Bargiotti, G.Castellani, P.Charpentier, J.Closier, U.Marconi, S.Paterson, R.Santinelli
Phone: R.Bernet, R.Graciani, M.A.S.Miguelez, R.Nandakumar
Apologies: N.Brook, A.Tsaregorodtsev, A.Smith
1. Production Status Report
Minutes from the production meeting are available here.
All signal simulation jobs have been submitted and are waiting in the queues. A 3.8K maximum number of
concurrent jobs was reached but this is still far from optimal. Joel reported that the SAM test suite was rerun on all
CEs and now ~60 are OK. There are still many permissions and mappings problems.
For the reconstruction, all jobs are submitted and some are running jobs remain for GridKa and CERN. Joel mentioned
that no jobs had been scheduled to GridKa recently and he is awaiting a response from the site admins. All stripping
jobs in the processing DB have been submitted and are running everywhere except GridKa. Many are failing in the
DaVinci step and this should be investigated ( ACTION: Joel )
Philippe conveyed that the sum of jobs obtained from the Bookkeeping is not consistent with expected numbers.
There is already some discussion of this on the mailing lists and Philippe will continue to investigate. ( ACTION:
Philippe )
Many jobs appeared to fail simultaneously at RAL and it was suggested that either SRM configuration or NFS access
was the culprit. Subsequently it was found that the software installation area had been corrupted due to incorrect
VOMS mappings of Raja's credential. The recent fixes to the DIRAC software distribution mechanism resolve this
situation (v2r15p1).
2. Production Operations Issues
Roberto reported that the creation of spurious '.lhcb' directories in site shared areas was due to incorrect VOMS
mappings of Raja's credential. The proposed solution from Joel and Stuart to the software distribution problems have
now been deployed in production. This prevents any user from writing to the shared area, regardless of their VOMS
mapping. The full software will be installed local to the jobs in the case that it is not found in the shared area. In
Joel's absence it was agreed that Stuart would take over the SAM software installation. Joel agreed to provide some
documentation for this and to rerun the tests at all sites before his vacation. ( ACTION: Joel, Stuart )
Philippe raised the issue of Castor mapping users to lcgprod in preference to other groups. For pool accounts there
must be a group associated to this or lhcbprod files would not be readable by the lhcb group. It was suggested that a
task force meeting with Tier-1 site representatives be arranged to discuss this issue. ( ACTION: Philippe )
Raja had problems generating a VOMS proxy of the correct size. As a result Stuart reported that we are now using a
Grid proxy in production. Roberto mentioned that the 1 month VOMS extension request has been refused and it was
unclear whether the gLite RB would renew credentials pending in site batch queues. Roberto agreed to investigate. (
ACTION: Roberto )
Two bugs were introduced to the pilot script affecting matchmaking of jobs and submission of generic pilot agents to
the production system, a partial fix was introduced by Stuart during the DIRAC3 week. Gianluca agreed to resolve the
use of glite-brokerinfo, where deployed. ( ACTION: Gianluca )
Several sites are deploying SLC4, since 32 bit compatibility is assured at eligible sites, LHCb will use it in this way for
now. For the future, the platform can be declared via the CMTCONFIG variable such that LHCb software tags can be
specified. Ricardo asked whether there was a convention for particular CEs to have 32/64 bit worker nodes and it was
confirmed that a homogeneous policy is recommended but not enforced. Philippe mentioned that LHCb can take
responsibility for publishing our software tags to reflect the WN architecture. The current plan is to make use of SLC4
native by August.
3. DIRAC3 Tasks
A template should be designed and circulated for the CHEP posters and talks. Philippe agreed to send a mail to the
developers list outlining a schedule of work. ( ACTION: Philippe )
Ricardo reported on the progress made during the DIRAC3 week, there were several discussions about the website
and developers were helped to set up DIRAC3 on their own machines. An important bug fix was made to the DISET
framework that was causing services to hang.
4. AOB
Nick agreed to chair the Operations meetings over the next 2 weeks while Joel is on vacation.
Raja mentioned that the RAL Castor LHCb instance is being deployed. There are 4/5 pools available and Raja was
asked to contact Nick about setting up the service classes relevant for LHCb.
Umberto reported that there is likely to be several months delay for the pledged resources due to the cooling
infrastructure not being in place. After a meeting with CNAF responsibles there is a possibility that the volume
currently used for STORM testing (~36TB) could be made available to LHCb.
There are minutes attached to this event.
Show them.