ScotGrid Technical Meeting

Wednesday 25 Jun 2014, 10:55 → 12:00 Europe/London

Other Institutes

Gareth Roy (University of Glasgow)

Scotgrid Tech Meeting 25 June 2014

 

 

-

David Crooks

Ewan Steele

Gareth Roy (chair)

Oliver Smith

Sam Skipsey (minutes)

Andrew Washbrook

 

-

Durham

-

 

Oliver:

One ticket: Issue with Durham ARC CEs having problems fetching LFNs.

Not sure what the problem was: they seem to be randomly fixing

themselves...

the issue is with validation of certs against CA certs.

 

[Is there a difference between the contents of the lcg-CA certs

package and the Nordugrid CA's package?]

 

Rebuilt disk server seems flakey. Runs for several hours then falls

over with "no resources left" for rfio (rfio processes replicate?).

 

Ewan: the plan for migration to new hosting - as soon as the

water-cooled doors arrive, we're going into D/T and moving the racks

(for the refitting of the doors etc), and then physical migration.

 

-

ECDF

-

 

Andy: opened up APEL ticket to resolve our legacy migration to EMI3

publishing. Our marked publishing records got wiped, so there's been

some to-and-fro trying to work out what happened. Dragging on quite a

bit - done all we can to republish on our end (seems to be an issue

with APEL's end).

 

System updates: cvmfs and kernel updates. Cvmfs should be fine - q:

can we do hot patches?

(We can hot patch from 2.1.7 to 2.1.19). This made things easier.

Kernel updates - middleware nodes fine. Blocker on the WNs (as ECDF

systems team have not updated (just patched) the kernel in some time,

due to the need for functional Infiniband kernel modules). Systems

team say things will be updated soon.

 

Andy noted that the issue with the systems team patching kernels

(rather than updating them) is that we get lots of false-positives on

pakiti.

 

Wahid dealing with WebDAV stability on DPM services.

 

 

??glExec policy??

 

Andy: if it's easy to install via tar ball (and systems team agree),

then we're okay with it. (Wahid has more objections from an ATLAS

perspective).

We're concerned about if the solution is designed for a shared system.

(But still waiting on tar ball!)

 

-

Glasgow

-

 

Dave: relatively quiet at the moment. Updated cvmfs, updating kernel

as we speak (just need the rolling reboots to push out).

 

Ongoing babysitting of disk servers.

Odd increase in Frontier load on our frontier squid - waiting for

ATLAS to add the second squid to frontier (was just CVMFS before). We

believe the load is correlated with the short-running recent Analysis

jobs that several sites saw high frontier load from recently.

 

Sam: (post-grid security and delegation mechanisms).

 

-

Ewan: (accounting) publishing still isn't working for Durham.

(Gareth notes that the APEL guys seem to be overworked at the moment,

it's probably best to ticket them, just to give them something to

remind them and prioritise)

Also need to check how our BDII publishing is working - we thought

we'd fixed it by having each ARC publish a fraction of the total

resource, but this doesn't seem to add up right either.

(Resolved via configuration management being fixed!)

-

 

 

Research Updates

--

 

[Gareth's geophysics recruitment story]

[Morgan's helping kettlebell world championship story]

Context: how much do we help out people in the wider world, which

might also help out the grid?

So, eg. can we learn anything from Andy's interactions with the ECDF

Systems Team? Can we learn anything from Ewan and Oliver's

water-cooled racks etc migration experience?

[What can we learn from all of our experiences outwith the grid]

 

Ewan asked Andy about his logstash experiences (this is still in

progress, motivated by the central university logstash policies).

(There is general ScotGrid-wide interest in logstash and other

analytics: Dave noted the interest in centralised logging at

HEPSYSMAN)

 

-

F2F planning? Everyone to think about if a non-management featuring

F2F is useful

-

 

 

 

Chat Log

--

 

Andrew John Washbrook: (25/06/2014 11:10)

https://ggus.eu/?mode=ticket_info&ticket_id=106140

post-GRID?

Is it all over?

Samuel Cadellin Skipsey: (11:20 AM)

Well, post-proxies.

(Central thesis: Proxies are the root cause of all of the problems

with grid authentication, and are what leads to horrible kludges like

glExec)

(It turns out if you go back to basics and design a solution based on

the actual security requirements of a distributed system, you don't

use proxies, and the entire system becomes both simpler and easier to

understand)

Andrew John Washbrook: (11:22 AM)

that sounds too much like common sense for day to day grid operations

Samuel Cadellin Skipsey: (11:23 AM)

Like so many suggested solutions(!)

Gareth Douglas Roy: (11:33 AM)

There is nothing quite like a deathly silence

Andrew John Washbrook: (11:33 AM)

thoughtful silence

how do we leverage our synergies

;)

pub quiz: how man lakes are in the lake district?

many

Samuel Cadellin Skipsey: (11:45 AM)

Fewer than you think, as most of them are meres and waters.

(for example, Windermere is a... Mere)

Andrew John Washbrook: (11:45 AM)

A: 1 (bassenthwaite)

11:00 → 11:15 Tickets

11:15 → 11:25 Research Updates

11:25 → 11:35 AOB