Choose timezone
Your profile timezone:
Deployment team
→
Europe/London
EVO - GridPP Deployment team meeting
EVO - GridPP Deployment team meeting
Description
- This is the biweekly DTEAM meeting
- The intention is to run the meeting in EVO: http://evo.caltech.edu/evoGate/. Join the meeting in the GridPP Community area.
- The phone bridge number is +44 (0)161 306 6802 (CERN number +41 22 76 71400). The phone bridge ID is 78425 with code: 4880.
Apologies: Raja, Derek, Duncan, Brian, Richard (& possibly Gareth), David (& Stuart)
Deployment team meeting – Tuesday 25th January 2011
Attendees:
Jeremy (chair+minutes)
Mark
Graeme
Pete
Wahid
Stuart P
Kashif
Daniela
Sam
Andrew
Alessandra
Stephen
Apologies:
Raja, Derek, Duncan, Brian, Richard (& possibly Gareth), David (& Stuart W)
Experiment problems/issues (20')
- LHCb
Raja is away until 8th February. Main reported issue is the continuation of job failures at Glasgow: http://tinyurl.com/6ldj7nn
SP wanted to check why the LHCb problem was first reported via the PMB. JC commented that the PMB reports are generally high-level mentions of ongoing issues and nothing of the technical detail is expected to be mentioned or discussed. The dteam meeting is the correct place for this discussion and while Raja is away it is perhaps inevitable that the matter is first noted during a PMB.
SP mentioned that the current job failures were unlikely due to the uploads failing. Put & register issues are more likely akin to the lcg-cr problems being seen by T2K. The latest update is in a GGUS ticket: https://gus.fzk.de/ws/ticket_info.php?ticket=66203.
- CMS
No update available.
- ATLAS
GS indicated that there was little to report at the moment. All sites are online. There is an ATLAS DB upgrade about to take place and that would lead to a drop in ADC activity.
Input from Brian
"1 - ATLAS have now migrated most of the data that they intend from MCDISK to DATADISK; and have deleted the majority of the files from MCDISK as well. If sites could again reduce the size of the ATLASMCDISK space token so that there is no more than 5TB of free space in the token; and to move the freed space into ATLASDATADISK.
GS added that any remaining data in MC disk is probably a replica and these will bed deleted by ATLAS shortly – as there are also copies in datadisk.
2 - ATLAS (uk) intend to go through and do another consistency check of all SEs.
- Cedric Serfon posted a link to a script for SToRM sites to create the syncat xml file format required by atlas.
- New dpm-toolkit 2.6.7 at http://www.sysadmin.hep.ac.uk/rpms/fabric-management/RPMS.storage/
WB: Toolkit available for a while. Brian has just discovered something useful and wishes others to use it.
SS: The pfn to surl tool for example which he used for Lancaster.
- Can sites please install ( new program which is helpful in cleaning up ATLAS[PROD.SCRATCH]DISK files older than a month is included in this version.)
Frontier squid changes.
PG: Setup was mentioned in an email from AF. General squid is easy to setup. Oxford did not use the ATLAS rpm.
JC: What are the main differences?
AF: ATLAS have customised things and the changes are not well documented. I set time aside to go through things and have twice written to ATLAS with a comparison and request to make it easier to follow. Main change is for frontier. One of the minor things is not being able to configure the override. They also added an option … why not use a normal squid? ATLAS suggest Frontier and they compile for frontier. It is for their convenience. I will go to their meeting again and mention issues directly – best way to feed back – such as using normal squid.
WB: It would be good to know if that’s the acceptable route.
PG: We use normal squid but had to set it up to work for ATLAS – Ewan worked on it.
PG: Do you want anything from ATLAS?
AF: I’d like a normal rpm – one that can be used without being concerned that something is wrong and worried about the configuration parameters
GS: So the request is to document things for the standard rpm. I’ll put in a Savannah bug. – Action
- Other
JC: I’ll be requesting a new VO to be setup for NA62. This is Dave Britton’s request. Initially the main interested institutes are Glasgow and Birmingham.
11:20
Meetings & updates (20')
- ROD team update
KM: There have been issues with GOCDB outages that have caused problems for the dashboard – alarms have gone to sites in downtime etc.
There is no concept of a pre-production site in the GOCDB so the current procedure treats the site as a production site. Tickets are created and alarms keep getting raised – it makes the country metrics look worse. Sites that are in the critical state can not have their alarms closed, even if they are testing a release.
JC will follow up – Action.
KM: The VOMS server at CERN has encountered problems following a 2hr downtime. The consequence being that we will soon not be able to renew a proxy for the Nagios testing.
- Nagios status (hardware update and plans)
JC: What is the current position with the purchase/install?
KM: The machine has arrived but is not complete – waiting for 4GB RAM.
PG: That has now arrived.
KM: The machine is not intended to act as a failover but will be used to install a WMS & SE as these components in the testing chain have shown problems. Nagios failover is under development and we’ll use it when available.
JC: Has John Gordon been in contact about testing the Irish sites?
KM: Yes. The problem is how to use one Nagios instance for two NGIs. No option yet in the infor.def file.
- Tier-1 update
No update was available.
- Security update
JC: There are no alerts for our sites in the latest Pakiti results.
-- T2 issues
CERN dteam VOMS was due to be switched off yesterday. No reports of problems from any sites.
- Escalated tickets https://gus.fzk.de/download/escalationreports/roc/html/20110124_EscalationReport_ROCs.html
58733 - Biomed at RALPP (dcache issue). Still on hold
64995 - RAL T1 No GlueSACapability defined for WLCG Storage Areas
65055 - Bristol Preparation work to enable ATLAS jobs at Bristol.
PG: They have other prioties at the moment .
- Return of the GridPP Technical Roadmap.
-- Still open for topics to be covered in addition to the usual: multi-core; GPUs and clouds.
11:40
gstat2 - publishing shares (10')
Input from Brian
"Tests at Glasgow show that much of the inconsistency in gstat is caused by space token setup. If for a dpm site , the site admin does not use SI short cuts (M,G,T,P etc ) then the space is matches to a better consistency.
IE, if allocating 12TB using the update-space command ; then this should be written as “12000000000000” and not “12T”"
SS: The change was to account in powers of 10 TB, but now the DPM is not fully allocating space because spacetokens are defined in powers of 2 TB. There is a translation issue somewhere.
AF has already identified publishing issues that affected NorthGrid sites.
- The correct shares to use are in Steve Lloyd's spreadsheet version GridPP3_T2_Allocations_2009 8.xls
Upon closer examination JC and PG agreed to discuss this offline before publishing more information/instructions on what is required to the sites.
11:50
Baseline versions and upgrades (05')
- Looking into upgrading (approx 5) DPMs which are below current WLCG BSWG recommendation of 1.7.4 to 1.8.0 ( glite3.2 version for SL5/64bit in current gLite release; Sam S. is preparing Glasgow svr018 for this upgrade. Wahid is testing the SL4/gLite3.1 version.)
- Next accounting period starts in April so this is likely the point by which sites will want to be stable. The only feedback from the PMB yesterday was to consider doing the upgrades before data taking restarts.
SS: the only major change from 1.7.3 to 1.7.4 is the clearing of the DB of transfers to stop growth causing problems. There is no reason to force this update. At some point sites will need to deal with the 1.8 schema change but 1.8 is not really sufficiently tested now.
WB: 1.8 has some changes. 1.7.4 changes are not dramatic so sites do not need to worry about it. 1.8 is still being tested so do not recommend using it yet. I am testing SL4/3.1 – there is still a memory leak that was supposed to be solved.
Conclusion: Sites should make changes before April if possible but being on a version greater than 1.7.2 is sufficient. Would only recommend the 1.7.4 upgrade for a subset of sites.
On the timeline, AM mentioned that the CB said their would be a clear announcement on the metrics to be used well in advance of the start of an accounting period. AF said there had not yet been any clear statement. JC/PG mentioned that they were being discussed and JC agreed to inform the PMB of concerns in this area.
11:55
Actions (05')
- http://www.gridpp.ac.uk/wiki/Deployment_Team_Action_items
12:00
AOB (01') Survey docword filepdf file
- EGI would like to launch the first SA1 survey covering the following
capabilities: Logging and Bookkeeping, Service Management and and
Auditing and gLite-CLUSTER. Please take a look at the attached document and send comments to John/Jeremy.
- Ireland has initiated the formal NGI validation process with the UKI ROC acting as the Integration Process Coordinator. The UK will make a request in the next 3-4 weeks.
EVO chat:
[11:00:54] Mark Mitchell joined
[11:01:35] Graeme Stewart joined
[11:03:01] Pete Gronbech joined
[11:04:13] Wahid Bhimji joined
[11:04:26] Mohammad kashif joined
[11:04:33] Daniela Bauer joined
[11:05:02] Sam Skipsey joined
[11:05:36] Andrew McNab joined
[11:06:10] Alessandra Forti joined
[11:18:04] Stephen Burke joined
[11:24:32] Wahid Bhimji Actually I should say that updating to the latest dpm-toolkit is generally a good idea - for bugfixes apart from anything else
[12:00:06] Mohammad kashif left
[12:00:09] Graeme Stewart left
[12:00:09] Stuart Purdie left
[12:00:10] Pete Gronbech left
[12:00:11] Alessandra Forti left
[12:00:12] Sam Skipsey left
[12:00:12] Andrew McNab left
[12:00:12] Mark Mitchell left
[12:00:18] Wahid Bhimji left
[12:00:23] Daniela Bauer left
There are minutes attached to this event.
Show them.