HTTP Deployment Task Force

Wednesday 29 Apr 2015, 16:00 → 18:00 Europe/Zurich

28/R-015 (CERN)

AGENDA

• Task Force - summary of motivation and consideration of the mandate
• Experiment and site plans for HTTP (very briefly!)
• Do the storage systems already provide the functionality required to make this exercise worthwhile?
  • Is there missing functionality which should be tracked by this group?
  • Should the WLCG middleware baseline take HTTP support into consideration?
• What monitoring solutions are preferred for tracking and validating deployments? How can sites validate their HTTP support?
• Are any changes needed to supporting services (for example configuration dbs and information systems?)
• How should we coordinate the supervision of the rollout?
• What other issues need to be discussed regarding HTTP and where should the discussions happen?

 

ADDITIONAL MATERIAL

• https://twiki.cern.ch/twiki/bin/view/LCG/HTTPDeployment

document HTTPTF.pdf

DECISIONS, ACTIONS, QUESTIONS

ATTENDANCE

Oliver Keeble (CERN IT-SDC)
Patrick Fuhrmann (dCache)
Christophe Haen (LHCb)
Xavier Mol (KIT)
Mario Lassnig (Atlas)
Cedric Serfon (Atlas)
Dave Dykstra (CMS)
Marc Caubet (PIC)
Hironori Ito (BNL)
Luca Magnoni (WLCG Monitoring)
Fabrizio Furano (CERN IT-SDC, DPM)
David Cameron (Atlas)
Andrej Filipcic (Atlas)
Alberto Pace (CERN T0)
Andreas Peters (EOS)
Alessandro di Girolamo (Atlas)

SCOPE & MANDATE

Mandate was not opposed, but will need to be refined in the light of the subsequent discussion regarding definition of a full set of requirements for HTTP storage.

FUNCTIONALITY

Decision - We can proceed with the deployment, but in parallel the TF should create a full checklist of the final desired functionality for HTTP storage services. Discussion scheduled for the next meeting.

Action - Oliver to create an initial table of functionality to be filled in by experiments and storage providers.

Action - Oliver to invite StoRM and xrootd reps to next meeting.

Question - we have to decide whether the TF is going to track the implementation of this stuff or simply leave the list as an output of the TF.

Question - incorporation of commercial HTTP storage, including S3. The more we extend our services, the more we will diverge from these systems. They will typically need some kind of gateway to handle authentication and metadata.  What is the TF position on this?

MONITORING and CONFIGURATION DBs

Decision - A common probe for validating HTTP storage is desired. IT-SDC group at CERN can maintain this, and indeed has created a Nagios/SAM probe for this purpose.

Question - which framework will schedule the tests? WLCG Dashboards can provide reporting.

Action - Atlas & LHCb to consider how best to schedule the tests. Can SAM be used or not?

Question - should performance testing be included here?

Question - when should the TF declare deployment complete? N% of candidate sites successfully passing the validations. N as yet undefined.

Question - For the test framework to schedule the relevant tests, it needs to access some authoritative list of sites/endpoints which have declared HTTP support. What should this be? GOCDB/OIM is the natural choice here. Proposal from Atlas involved using GOCDB/OIM info to find an endpoint and then fetching a standard file from a well-known location which contains all relevant data about the storage system.

Action - Atlas and LHCb to consider the options.

Traffic monitoring was also discussed. Requirements for this will be taken into account in the broader discussion of necessary features for HTTP storage (next meeting)

Decision - tackle the monitoring and validation questions first before getting into detail about how the rollout should be tracked.

NEXT MEETING

Next meeting scheduled 3rd June, same time (16hr).