PASTE meeting
2-R-014
CERN
PASTE Meeting, 7th September 2010, CERN
Chair: P.Charpentier
Minutes: F.Stagni
CERN: Z.Mathe, F.Stagni, S.Paterson, A.C.Smith, V.Romanovskiy, M.D.Rashid, R.Santinelli, P.Charpentier, J.Closier
Phone: R.Graciani, A.Zhelezov
0. Development tasks:
0.1. Bookeeping
For the last week we've no progress on the merging productions, and stopped the ongoing reconstruction. RAL problems also stopped us from completing. 90% of the jobs failed when checking for descendents in the bookkeeping. A lot of calls to the getDescendents() method causing the service to stuck. Zoltan says that most of the calls come volhcb20, and he's suggesting a possible bug with the DataRecovery agent, Andrew thinks that it's not possible. Anyway Stuart stopped for 10 minutes the DataRecovery agent to see if it is causing the excessive load, but this does not seem to be. Zoltan will have a second look. Another experiment is to restart the bookkeping service to see if the number of threads still increase: this seems to be true, because after 5 minutes we already had 27 threads running. It could be a wild loop in bookkeeping code.
0.2. RSS activation
We decided to activate a real ban (and un-ban) of the sites and SEs, for the following cases:
- an OUTAGE downtime on the site is announced: 12 hours before it starts, the site in banned
- an unscheduled OUTAGE downtime on the site starts
- when all the CEs of a site are in OUTAGE downtime, the site is banned (12 hours in advance, if possible)
- when the SRM interface is OUTAGE donwtime, the corresponding SEs are banned, just when the actual downtime starts.
- when a SE has 0 space left, then it is banned for writing.
Some basic management will be put on CS side.
It will be investigated the possibility to use the notification system for eLogg-ing these actions.
0.3 Throttling
Job throttling is done and tested, will be in the next release.
1. Production Issues
CondDB problem. We now persist the information in the CS. Information about it should be forwarded to the lhcb-grid ml so that GEOC could update that.
2. DIRAC Release Update
Some fix may require a patch release.
For what regards new releases that should pass the certification tests, nothing too urgent, so we'll wait for Andrei to be back for that.
3. DIRAC Tasks
Nothing major to report.
4. AOB
NTR