Choose timezone
Your profile timezone:
PASTE meeting
→
Europe/Zurich
2-R-014 (CERN)
2-R-014
CERN
Description
CERN, bat 2 R-014
To join the telephone conference, click here: https://audioconf.cern.ch/call/0165202
To dial in to the conference:
a. Dial +41 22 7676000
b. Enter access code 0165202
Production and PASTE Meeting, 12th May 2009, CERN
Chair: A.Tsaregorodtsev
Minutes: A.Tsaregorodtsev
CERN: Z.Mathe,F.Stagni,J.Closier,A.C.Smith,R.Vazquez,M.Seco
Phone: R.Graciani, A.Tsaregorodtsev
0. Ongoing Action List
(94) - 23/09/08 - Andrew, Marcos - Using SystemLogging service for dynamic services status monitoring (Andrew to provide FTS example)
(111) - 14/09/08 - Alexei,Andrei - Productions with file merging and associated data distribution implementation
(116) - 18/11/08 - Ricardo - steer the CPU related activities in DIRAC, document procedures, move to the HEP CPU units
(122) - 13/01/09 - Ricardo,Vladimir - Develop specialized SAM Agent to track and report errors with regard to pilots
(132) - 03/03/09 - Stuart - Update LFN construction utility to store production output directories as a parameter
(133) - 03/03/09 - Matvey - Provide request DB interface page
(139) - 31/03/09 - Roberto,Stuart,Ricardo - Track and test glexec as it is deployed at the sites, put CE into production pending the official release
(140) - 31/03/09 - Andrew - Formulation of procedure for how to treat banning of sites based on job requirements e.g. input data / staging...
(143) - 14/04/09 - Andrei - update the links in the job monitoring info sent to SiteMonitoring
(144) - 21/04/09 - Zoltan - propose and implement solutions to reduce the Bookkeeping service load for production input data queries
(145) - 21/04/09 - Vladimir,Roberto,Joel - check why SAM application test jobs are running by sgm user on some sites
(146) - 21/04/09 - Vladimir,Joel - SAM SW installation logic by synchronizing shared areas with the CS SW specification.
Should be complemented with the installed SW monitoring tools.
(147) - 21/04/09 - Joel - renew the request to IT for deployment of the new LHCb/DIRAC central host machine
(148) - 21/04/09 - Joel - setup DIRAC SVN/Trac project portal and configure access to the DIRAC developer team.
(149) - 05/05/09 - Ricardo,Andrew - test the --use-grid SetupProject option to set the LFC_HOST to be used by CORAL with Merging jobs
Actions closed:
(142) - 14/04/09 - Adri - verify that the data transfer monitoring info sent to SiteMonitoring is functioning
(150) - 05/05/09 - Roberto - update the Production Procedures TWiKi with details on GGUS ticket submission
1. Action Updates
- Action(94) - Logging system usage examples
Ongoing. This will be one of the discussion topics during the coming DIRAC week.
- Action(111) - Production definition tools
The Production Request Web interface is complete. It needs independent tests and documentation. Joel will test it
and help documenting associated procedures.
Alexei is now preparing a real production example prepared using Production Request information and templates
editable with the Workflow Editor.
- Action(116) - CPU normalization studies
Ongoing. Ricardo reported that jobs are now reporting the CPU time consumed in the same units as given by sites -
500 SI2K. Analysis of the benchmarking tests is going on. Then the jobs will be reporting also in the new HEP SI
units as well.
- Action(122) - Analysis of the SAM pilot submission errors.
Ongoing. The errors are now doen manually by Vladimir but eventually an automatic analysis should be possible.
Logging service should be used to accumualte these errors with subsequent analysis.
- Action(132) -
Ongoing with lower priority
- Action(133) - RequestDB monitoring
Ongoing.
- Action(139) -
Ongoing, pending more sites deploying glexec. Roberto made some basic tests at IN2P3, problems were found.
Ricardo has volonteered to do tests with real jobs.
- Action(140) - Banning faulty storage elements
Ongoing.
- Action(142) - data transfer monitoring info sent to SiteMonitoring
Adri has sorted out the problems in contact with Elisa. Action closed
- Action(143) - links in the job monitoring for SiteMonitoring
Ongoing.
- Action(144) - Bookkeeping query optimization for the production management
Ongoing.
- Action(145) - Check user roles for the SAM App test jobs at sites
It was verified that the jobs are sent with proper credentials. However, there are sites not properly
configured, e.g. using gridmap file mapping prior to the VOMS mechanism. Once discovered, GGUS tickets
are issued to those sites.
- Action(146) - SW installation framework using CS data
Ongoing.
- Action(147) - new DIRAC service host
The host OS should be updated to SLC4. The machine is expected this week.
- Action(148) - SVN/Trac server
Ongoing.
- Action(149) - use of --use-grid option
Ricardo reported that the first test of GaudiApplication with the --use-grid option of the SetupProject
was OK. However, this was doen with the MC workflow and did not involve reading input data from the grid
storage. This makes the test imcomplete. It was agreed to perform more tests with jobs with input data,
for example, Merging production jobs (Action Andrew)
- Action(150) - Update TWiKi with GGUS procedures
Roberto has updated the TWiKi. Action closed.
2. Production issues
- Production status
First 10M events of MC'09 production are done and merged. The events are now being analyzed
by the users. As was pointed out by Philippe, the SIM and DIGI files for the first portion of
jobs were not stored. This fuctionality has to be restored.
A new patch release v3r14p3 is being prepared to contain CMT environment fixes and a possibility
to check out of the input data metadata in the WNs to speed up the job input data checks.
It was discussed that the errors reported by the applications to the datacrash mailing list should
be first analyzed and filtered by te shifters before passed to the application managers.
3. DIRAC Tasks
- Infrastructure monitoring (SAM,SLS,Dashboard)
No update
- Framework status
- Bookkeeping status
Zoltan reported problems with accessing the Dev Oracle service due to changed password problems.
This was later related to a faulty service attempting to access Oracle with a wrong password.
He is preparing a schema update to allow data interrogation for data registered in a given time
interval in order to optimize data queries for the Production System.
- WMS status
Rashid reported on his work on the Online Computing Element. He has prepared the schema of
software components to be developed as was discussed during the last DIRAC week. In order to
proceed with the implementation, Andrew will help Rashid to deploy a DIRAC installation for
this development. Andrei asked Rashid to document also the agreed deployment schema to help
the develop,emt and discussion with relevant people.
- DMS
A number of small fixes is being finalized for the next release.
- Production Management status
See Action 111
- VO-boxes
No update
- Web Portal
No update
- Services deployment ( hardware, configuration )
See below
- Documentation
No update
4. DIRAC hardware configuration.
Ricardo presented a proposal to deploy DIRAC services for the near future. See the document
attached to the agenda page. This proposal aims at the short time perspective and counts on the
already available hardware resources. Several remarks were made. Joel suggested to consider the
the CERN SysLog facility to store the data recorded by the Security Logging Service. The advantage
is ensured backup of the CERN facility.
The proposal is short term and does not contain solutions to the database backup, redundant services,
load balancing means. This needs further studies and will be discussed during the next DIRAC week.
The experience of ATLAS with the PanDA services deployment at CERN should be considered. Andrei will
invite an ATLAS expert to present their experience during the DIRAC Week.
5. AOB
The next DIRAC week will take place the week of 8th June at CERN
There are minutes attached to this event.
Show them.