Production and PASTE Meeting, 14th August 2007, CERN
Chair: A.Tsaregorodtsev
Minutes: S. Paterson
CERN: M.Bargiotti, A.Casajus, G.Castellani, P.Charpentier, J.Closier, R.Graciani, G.Kuznetszov, A.Smith, S.Paterson, A.Tsaregorodtsev
Phone: R.Bernet, F.Donno, R.Nandakumar, M.A.S.Miguelez
Apologies: N.Brook, R.Santinelli
1. Production Status Report
Joel reported that investigation with Markus and Philippe revealed a problem with DC-06 stripping jobs. A mixture of
b-inclusive and minimum bias datasets were being stripped in the same job so it was agreed to restart the
reconstruction activity from scratch. O(10K) jobs are to be submitted and concerns were raised over the status of
some Tier-1 sites, GGUS tickets should be submitted as necessary when problems are found.
Stuart mentioned that several jobs had been identified by the Job Sanity Agent for which output data already exists.
The production WMS should be regularly checked for this type of failure and the appropriate actions performed in an
automatic way. Andrew agreed to pass the list of input data for some failed jobs for Marianne to correct. ( ACTION:
Andrew, Marianne )
There was some discussion regarding replicas for job output data. The consensus was to modify the existing
workflows to store replicas at CERN + 2 Tier-1 sites. ( ACTION: Joel )
DIRAC v2r15p3 will be put into production for the DC06 rerun. Andrei will put fixes in place for the logic in the
transfers. ( ACTON: Andrei )
After the recent upgrade to volhcb02 a DISET authentication problem has been observed in the CS. Adri agreed to
investigate. ( ACTION: Adri )
2. Production Operations Issues
- Data integrity checking
There is currently no BK <-> LFC checking in place. Andrei remarked that this should be performed continuously as a
DIRAC Agent and that the procedure should be revised. ( ACTION: Marianne )
- SAM test system update
Joel reported that the list of sites eligible for LHCb is now extracted from the SAM DB immediately prior to job
submission. The daily cron job for SAM tests is now also in place.
- SRM2.2 tests status
Andrew gave a summary of progress with the GFAL python binding. Some minor problems have already been
reported back to the developers. The current implementation of GFAL appears to be incompatible with our intended
use-case. For example, when pinning a file, a GFAL internal object must be persisted for the duration of the pin or it
becomes impossible to 'unpin'. This work is ongoing. ( ACTION: Andrew ) Some development remains on the DIRAC
side which will occur pending the GFAL integration. ( ACTION: Stuart )
Flavia conveyed that a new twiki page is being kept up to date with the latest problems and solutions for SRM v2
testing. Flavia also mentioned that the removal of port numbers in LFC replicas be performed for us and Andrew
agreed to submit the GGUS ticket request. ( ACTION: Andrew )
- Job priorities calculation
Gianluca reported that MOAB was starting to be used to test jobs using the 'Jobs.txt' file to supply the current system
state.
- volhcb02 Upgrade
volhcb02 is to be upgraded requiring a reboot and it was suggested to schedule this for next Monday. ( ACTION: Joel )
3. DIRAC3 Tasks
DIRAC3 development is ongoing.
4. CHEP
It was agreed to reconvene at 2pm to discuss the available draft contributions for CHEP. The LHCb contributions are
available on the following
agenda page.
5. AOB
None
There are minutes attached to this event.
Show them.