Operations team & Sites

Tuesday 20 Dec 2011, 11:00 → 12:16 Europe/London

EVO - GridPP Operations team meeting

- This is the biweekly ops & sites meeting - The intention is to run the meeting in EVO: http://evo.caltech.edu/evoGate/. Join the meeting in the GridPP Community area. - The phone bridge number is +44 (0)161 306 6802 (CERN number +41 22 76 71400). The phone bridge ID is 115728 with code: 4880. Apologies: Elena

11:00 → 11:05 Networking update

11:05 → 11:15 Experiment problems/issues

Review of weekly issues by experiment/VO - LHCb - CMS - ATLAS * CVMFS transition almost complete left todo Brunel and Cambridge are validated and have to syncronise their tags Birmingham will start validation today * GROUPDISK pledges have changed and Glasgow has been asked to increase by 100TB and Oxford by 50TB by January. * There are still problems caused by the CA certificate. CERN said they'd fix it but this doesn't work yet. Elena has already wrote to Steve Traylen and the AMOD Atlas top shifter. As personal remark on this I think the UK CA certificates have created during the past years enough deployment problems whether because of change of DNs or because of the email address and this should be frankly discussed by the PMB since so far the administrators haven't been listened to (I have had grudges about these changes since 2006). other than that Merry Christmas! - Other - Experiment blacklisted sites - Experiment known events affecting job slot requirements - Site performance/accounting issues - Metrics review

11:15 → 11:29 Meetings & updates

- ROD team update - EGI ops Peter's analysis suggests a lot of UK sites still using CERN. You'll see from yesterday's mail that CERN are asking sites to desist. You can see from Peter's list attached that some UK sites are using CERN as 3rd on their list but several are using it as second. However, I see several sites who ref CERN also use lcgbdii.gridpp.ac.uk which I can't see in the DNS. Is this an old name? I thought it might be a typo for lcg-bdii.gridpp.ac.uk but some have both defined. My reading of the order in col C is that:- Glasgow will use CERN all the time, and don't use Scotgrid bdii. Bristol will use CERN all the time, but do use Scotgrid bdii None of these are using Imperial as we recommended. Can you ask them all to remove lcgbdii and CERN. I'd be interested to know if having lcgbdii as the first in the list will be detrimental to the sites performance. - Nagios status - Tier-1 update - Security update - VOMS/VOMRS problems for users Due to the CA issuer name change and checks in some VOMS instances, users moving over to their new certificates have been encountering problems getting approvals through. Manual editing has been the workaround but now Steve T has given a method for a bulk update. All UK CMS users were dealt with last week and with no problems found this will happen now for the other VOs. For a background update on this issue see: http://nationalgridservice.blogspot.com/2011/12/ca-stuff.html. -- T2 issues Plans over Christmas -- General notes. - A reminder to please update the https://www.gridpp.ac.uk/wiki/HEPSPEC06 page - GDB.... we'll review it in the next meeting. - Tickets Overview: Down to 30 tickets this week! A number of tickets were caused by bad lsc files, as the threads on TB-SUPPORT have discussed there's got to be a better way! If a user is going off on holiday for a few weeks is the correct Ticket state "On Hold" or "Waiting for Reply"? A request - can everyone check their tickets before they switch out the lights and head off for Christmas. The interesting ticket shortlist: 77529 - Hone auth error at the Tier 1 WMS. Could be similar problems to what IC saw in ticket 77529 76996 - Atlas missing library problems. Has anyone seen wierd behaviour caused by tarball mounts being overly stressed? 76119 - Biomed asked Manchester to clean up their space for them. Interesting implcations here. RALPP tickets - still on hold. Did Chris forget about them? :-) The longlist: QMUL: https://ggus.eu/ws/ticket_info.php?ticket=77460 Biomed having trouble with an SE they shouldn't really be using (QMUL's test SE). Chris wants to have a look at the publishing, but it's low priority. https://ggus.eu/ws/ticket_info.php?ticket=77463 Is a duplicate of 77460? IC: https://ggus.eu/ws/ticket_info.php?ticket=77529 Hone auth error. Can be closed. Included today because of 77528. Tier 1: https://ggus.eu/ws/ticket_info.php?ticket=77528 Hone again, have WMS submission troubles. Could it be similar to 77529. Birmingham: https://ggus.eu/ws/ticket_info.php?ticket=76996 "Missing library" problems for Atlas. Mark, Wahid and myself have been discussing this offline and think it might be related to shared nfs mounts for our tarballs. Lancaster has moved to its own exclusive nfs server for its shared cluster. RallPP: https://ggus.eu/ws/ticket_info.php?ticket=76841 Is Chris back? The ticket is still on hold. Zeus user having trouble accessing storage (although it seems to occasionally work). RalPP's other tickets are also "On Hold" still. Manchester: https://ggus.eu/ws/ticket_info.php?ticket=76119 Biomed reopened the ticket and asked Alessandra if she can track down greedy biomed users on the Manchester SE. Interesting as they've now asked her to delete files. Shouldn't cleanup be their job (for liability reasons as much as anything else. Durham & Glasgow: https://ggus.eu/ws/ticket_info.php?ticket=75488 (Durham) https://ggus.eu/ws/ticket_info.php?ticket=75320 (Glasgow) These two linked tickets can probably be set to "Waiting for Reply" as the users are off on their Christmas Vacations. Everyone: Have a good Holiday and a great New Year!

BDII list sites-cernbdii-better.xlsx.xls

11:29 → 11:30 AOB

- Next meeting 10th January 2012 - Have a great Christmas everyone and Happy New Year!