GridPP Cloud project meeting 1st March 2013

Contents

Present
Actions from the previous meeting
Experiments
Imperial GridPP pilot
Other UK cloud sites
Connections to other projects
Security - John Green
AOB
Actions

Present

David Colling, Pete Gronbech, Andrew McNab, Kashif Mohammad, Peter (?), Wahid Bhimji, Matt Doidge, Robert Frank, Chris Walker, Andrew Washbrook, John Green, Linda Cornwall, Robin Long, Andrew Lahiff

Actions from the previous meeting

David to report on EGI conference call
1. Agreed to join
2. Not started yet
3. Appears to be very much "grid on cloud" e.g. uses GLUE2
4. We will be involved in their various demos
5. David asked for Kashif's experiences:
6. Kashif said he had some involvement but not much
7. A cluster was installed at OeRC, though it wasn't part of the federated cloud
Adam to create a store area for Andrew - DONE
Andrew to send link for dashboard - DONE
Documentation of the Imperial-based pilot on the wiki - STARTED, ongoing
Wahid to write a plan for storage testing - DONE
Cloud community - ongoing
David to distribute a link to GDB discussions - DONE

Experiments

LHCb - Andy McNab

LHCb DIRAC has agreed to use VMDIRAC, when it's more mature
Mario has access now to the GridPP cloud at Imperial
Experiments in Manchester with virtualization and CernVM image
- E.g. reducing the number of cores and RAM in the VM, while it's running
- Specify a shutdown time using HEPiX protocol e.g. if rack maintenance is to occur in a week's time
- As jobs expire and the VM uses less CPU time, reduce the number of cores it's using
It is possible to have a shared filesystem between the hypervisor host and VM e.g. for contextualization
Want to ensure APIs are identical between standalone and cloud images
Using CernVM because HEPiX tools are installed
Some discussion of Boxgrinder - MAJH to put documentation on wiki
CernVM to move away from Conary in version 3

ATLAS- Peter

BNL and CERN have working cloud setups
Testing on Imperial setup
Also looking at RAL and ECDF
Want to spin up lots of instances
HTCondor is the core component on which they're relying
Need to understand the tools better e.g. Boxgrinder, cloud-init etc.
ACTION: Adam to put notes about Boxgrinder etc. on the wiki
David commented that there is a general push for common submission tools across the experiments

CMS - Andrew Lahiff

Local storage access fixed the CVMFS problems
Jobs are running well now
A CVMFS race condition has been fixed
VMs are still 'hanging around' in OpenStack i.e. taking up resources in 'shutoff' state
Andrew has provided log files to developers - this is the main issue to be resolved at the moment
Andrew suggests more stress testing as the next task - More than 100 jobs, for example
Similar issues on the HLT - Need to convince HLT people to improve the networking

Imperial GridPP pilot

Now 7 compute nodes with 200 cores and 400 GB RAM
Swift storage node partially commissioned
Installed Cloud Scheduler, which had been mentioned in a previous meeting
IC Networking team had concerns about permitting external access via floating IPs, so we're in discussions with them
Kashif asked which networking system we're using - Using nova network
ACTION: Adam to provide a gateway node for access, while the floating IP issue is resolved
Peter asked why Cloud Scheduler was being tested? - ATLAS centrally decided not to use it, preference for HTCondor now - Peter will contact Victoria University people to ask them to contact UK clouds - Chris Walker asked - is Cloud Scheduler useful for smaller VOs? Easier to use than HTCondor?

Other UK cloud sites

Connections to other projects

Security - John Green

Has added page on wiki, linked from useful links page
Managing images, keeping them up to date
JANET, legal etc.
Maybe more exposed to vulnerabilities than grid software, because more widely used
Firewalling
VM segregation
Asked for further suggestions
Traceability
Root access - External access requirements should lower over time, as move to Production
He will respond to and look into particular concerns if people raise them with him

AOB

David asked if anyone was planning to submit a paper to CHEP? - CMS paper about HLT and national work - Maybe Lancaster - Some ATLAS ones too, according to Wahid
Kashif mentioned a free Amazon meeting, for which he will circulate the link

Actions

1. Adam to provide a temporary gateway node for external access to instances

2. Adam to put notes about image creation and other relevant tools on the wiki