Choose timezone
Your profile timezone:
PASTE meeting
→
Europe/Zurich
Description
CERN, bat 2 R-030
To join the telephone conference, click here: https://audioconf.cern.ch/call/0122882
To dial in to the conference:
a. Dial +41 22 7676000
b. Enter access code 0122882
Production and PASTE Meeting, 8 May 2007, CERN
Chair: A.Tsaregorodtsev
Minutes: A.Tsaregorodtsev
CERN: J.Closier, R.Santinelli, P.Charpentier, A.Tsaregorodtsev, R.Nandakumar,
M.Sapunov, G.Kuznetsov
Phone: U.Marconi, N.Brook, R.Bernet, R.Graciani
1. Production status ( J. Closier )
Monte_carlo prodution will restart at the end of the week or next week with the new
Gauss version.
Reconstruction. Running now at RAL, IN2P3, CERN. The observed problems are:
- PIC: misbehaving SRM resulting in too long response times. ACTION: Ricardo
will pass the message to the service provider and investigate the problem if
necessary;
- GridKa: slow staging to be investigated. Most likely the storage system dimensioning
is not adequate. GGUS ticket sent with a copy to M.Schmeling;
- CERN: some staging requests are never finishing due to some tapes not in the tape
queues. ACTION: Joel to investigate.
At RAL the files in the dCache storage presumably lost were in fact recovered and should
be now reregistered in the LFC ( ACTION: Marianne ).
No more reconstruction jobs left for CNAF and NIKHEF.
Stripping. There is a problem at Lyon of loosing connection with an opened file in the
dCache storage. The dCache library reports the connection to be restored but the
application fails nevertheless. This happens to large files with long open times which
might be a hint to the problem. Adria has contacted David and Lionel at Lyon on this
subject. The problem might need intervention of the ROOT and POOL experts to clarify
the details of the dCache/ROOT/POOL interactions.
Stripping will now proceed with the new versions of the Vrunel ( v30r17 ) and DaVinci.
The Brunel version fixes the Calo event size problem. Now it produces ~1GB file per
one input rdst file. It was agreed to limit the number of input rdst files to 2.
DaVinci is sometimes crashing which can be reproduced in a standalone run - Patrick
will investigate. Patrick's EMail will be added to the DaVinci(Stripping) workflow
to send messages about the crashes.
Nick reminded that the status of the SRM storage stability is not sufficient for our
operations and this should be reported to the highest level of the WLCG management
bodies. It was agreed and Roberto, Andrew and Stuart will provide plots with a
quantitative evidence of slow responses of the SRM end-points in use as a function of
time to illustrate the situation and make it possible to correlate the variations
of the SRM behavior with activities outside LHCb. It was also suggested to upgrade
the SRM tests to use srm-get-metadata instead of srm-ping in the critical part of the
tests. The SRM critical SAM tests should run at once per hour frequency. This problems
still should be reported to the operations meetings ( Roberto ) with the regularly
produced SRM latency plots. The collaboration with site managers should continue to
understand the problems and to correlate them with other on-site activities. A SAM
test will provided to test the SRM's to have an LHCb view of the SRM statuses ( ACTION:
Joel ).
Raja reported on the RAL dCache to Castor migration plans. Starting from the end of
June the dCache will be set to read-only and new files will be only written to Castor.
We can start to write to Castor with a RAL_Castor-tape SE definition which will be later
renamed to RAL-tape. A dedicated workflow to test the RAL Castor with the output
specified to go to RAL_Castor-tape will be prepared ( ACTION: Joel ). After all the
successful tests of the Castor storage the remaining files in the RAL dCache will be
copied to Castor.
2. Stager service on the User system.
Ulrik has reported that his jobs are stalling on the User system after the Stager
service was put into operation there. The problem is to be investigated by Stuart and
Andrew ( already solved by the time of writing minutes ). However, it is clear that more
elaborated algorithms should be investigated for the cases where a job has multiple
choices of the destinations sites.
3. CERN_CastorDebug SE.
The CERN_CastorDebug is defined now as a GRIDFTP storage with the backend in Castor.
This proved to be a useful tool for debugging the problems that happened in the
applications running in the grid jobs. Now we have got more than 5k files in this
stirage element ( more than 1 TB) which are not registered in the LFC and not used
any more but there is nobody reponsible for the clean-up of this area. It was agreed
that Joel will check with Marco, Gloria and Patrick what is the best way to provide
input to the applications to be debugged. Then the procedure to clean the SE will be
also defined.
4. DIRAC3 developments
Ricardo reported that the work on the DISET framework is close to the first working
prototype that can be used by the services developers. More work is still necessary on
the logger client and server. More work is needed on the services authorization
mechanism to make it compatible with the new Configuration schema. The Configuration
service is operational although there is now client to update it yet and the updates
should be done by hand in the configuration file. The work on these components is
continuing.
The volhcb03 machine is reinstalled with SLC4 and is ready to be used for the DIRAC3
testing. The DIRAC3 build and deployment tools are now urgent to be put in place. Joel
and Ricardo will get together to come up with a detailed proposal of the tools and will
try the tools on the volhcb03 machine next week ( ACTION ).
The WMS databases schema and corresponding front-end classes are ready with the
functionality similar to DIRAC2. The handlers for the JobReceiver service is ready to be
tested pending the first DIRAC3/DISET release. Other handlers will follow shortly.
( ACTION: Andrei ). All these will need prototyping on the volhcb03.
The new workflow description classes are prepared by Gennady. He will now produce a
simple module example to illustrate the use of the new framework. After some possible
discussions and iterations that should take place this and next week, the work on
converting the existing modules will start ( Joel, Stuart, Gennady ). For the time being
the workflow will be generated in a scripting environment. The corresponding editors
will be produced as a deep rewrite of the current editors ( migration from C++ to python
).
4. AOB.
On the next PASTE meeting we will discuss the LHCb VO user support that will be
presented by Maria Demou-Zacharova. The material to be discussed is attached to the
next meeting agenda.
Another point for the next meeting is a discussion of the way how to define the LHCb VO
policies concerning the use of the common resources. The discussion material is
prepared by Gianluca and was already circulated in the developers mailing list.
There are minutes attached to this event.
Show them.