Choose timezone
Your profile timezone:
PASTE meeting
→
Europe/Zurich
Description
CERN, bat 2 R-030
To join the telephone conference, click here: https://audioconf.cern.ch/call/0122882
To dial in to the conference:
a. Dial +41 22 7676000
b. Enter access code 0122882
Production and PASTE Meeting, 17th April 2007, CERN
Chair: S. Paterson
Minutes: S. Paterson
CERN: M.Bargiotti, G.Castellani, J.Closier, A.Smith, S.Paterson, R.Santinelli
Phone: R.Bernet, N.Brook, C.Cioffi, G.Kuznetsov, M.A.S.Miguelez, R.Nandakumar
Apologies: P.Charpentier, A.Tsaregorodtsev
1. Production Status Report
Minutes from the Monday meeting are available here. Joel reported that simulation jobs were running smoothly and
all requests had been satisfied. Reconstruction jobs have now run successfully at all Tier-1 sites. Some problems with
the Stripping workflow have been identified. M.Cattaneo pointed out that Brunel seems to open and close the .digi and
.rdst files at each event and provided a solution for this. P.Koppenburg highlighted a problem with the DaVinci
workflow selecting random events which should be understood ( ACTION: Joel ). In order to proceed in line with the
DC06 cleanup Stuart suggested to remove all jobs in the "staging" status and submit jobs to test the new workflow
(the StagerAgent will purge the StagerDB automatically). Raja agreed to delete the "staging" jobs. ( ACTION: Raja )
Roberto raised the issue of DCache files not being written to the file namespace potentially resulting in lost data.
A.Casajus' script is still running in order to work out the extent of the problem (seemingly associated with high loads
on the system) and will present a list of lost files when it completes. It was agreed that this issue should be escalated
to the DCache developers as soon as the data is available. ( ACTION: Adri )
2. Production Operations Issues
Stuart reported that DIRAC v2r14p3 was released last Friday to cope with the Castor1 TURL problems at PIC, this is
now running smoothly.
Andrew sent out a detailed plan for the DC06 cleanup:
1) Extract list of LFNs for b inclusive '.digi' produced with correct Gauss version from bookkeeping (Carmine) (17 April)
2) Produce list of replica for these 'digi' for ProcessingDB. Use replicas outside of CERN where present. (Marianne) (17/18 April)
3) Complete regeneration of ProcessingDB (Andrei/Andrew) (17 April)
4) Regenerate transformations/workflows using a new configuration (Joel) (17 April)
5) Repopulate ProcessingDB replica information (Marianne/Joel/Andrew) (18/19 April)
The list of Actions was agreed by all responsibles. Joel confirmed that four new transformations will be created
(B-Inclusive and Maximum Bias both at high and low luminosity). Carmine mentioned that it may take several days
to update the Bookkeeping views but the other Actions should be completed on time. Marianne commented that it
would take until Friday to determine the list of files but Roberto suggested using the CNAF RO instance of the catalog
directly to optimize this. ( ACTION: Marianne). This may still allow the reconstruction/stripping activities to be started
by Thursday. Stuart commented that several optimizations will be made to the StagerAgent and StagerDB during this
period of downtime, leveraging on experience gained so far. ( ACTION: Stuart, Andrew )
Regarding the data accumulation at CERN, Andrew reported that all files that could be transferred have now been
transferred. There are remaining files at PIC and NIKHEF which correspond to the problematic files as a result of
DCache not writing files to the namespace. More news will follow pending the results of A.Casajus' script.
Andrew also gave a summary of his finding from the zero-size file investigation. What seems to happen is that the
LFC registration operation fails after the LFN is registered but before the size is entered. The Replica Manager
correctly picks up this error but subsequent transfer and register operations will fail due to the LFN already existing.
The solution to this is that in the event of a registration error, the LFN is removed from the catalog before the
registration request is put to the VO-box. Andrew agreed to implement the fix. ( ACTION: Andrew ).
Following last week's Action, Gianluca is modifying LCG.py and is almost ready to deploy it wherever Grid commands
are used ( ACTION: Gianluca ). After successful testing, the User system can be migrated to volhcb01 and we can
announce to the experiment the switch to VOMS proxies.
The issue of why pilots submitted to the Test system cannot reach LCG.RAL.uk is still not understood, Roberto
volunteered to investigate. ( ACTION: Roberto )
3. DIRAC3 Tasks
Due to the absence of Andrei and the connection problems of Ricardo, the Framework status was not reported. The
Building Tools status is unchanged from last week due to the Easter break.
Gennady asked about the status of lxgate14 and it was unclear whether the upgrade to SLC4 has been requested.
Joel agreed to look into this. ( ACTION: Joel )
4. AOB
Roberto announced that STORM has been undergoing testing at CNAF. So far the full features of SRM v2 have been
tested and Roberto confirmed the results so far are encouraging. The next step is to stress test the system with
LHCb analysis jobs.
Nick asked what the status with glexec is, given that IN2P3 has agreed to participate with the testing. Stuart
mentioned that the Stripping/Reconstruction activity and the Stager have taken top priority until now but after a
successful restart later this week perhaps some progress can be made. In terms of DIRAC development required to
make use of glexec, most of the infrastructure is in place. A proxy delegation service and glexec aware DIRAC CE are
the only missing components. ( ACTION: Stuart )
There are minutes attached to this event.
Show them.