DPM white paper kickstart meeting

Friday 13 Sept 2019, 09:00 → 12:00 Europe/Zurich

Frederique Chollet (Centre National de la Recherche Scientifique (FR))

Presents:
OK: Oliver Keeble
BG: Baptiste Grenier
AD: Alessandra Doria
TK: Tomoe Kishimoto
MK: Michira Kaneda
RS: Ruy Sawada
PV: Petr Vokac
AS: Andrea Sartirana
FC: Frédérique Chollet
MJ: Michel Jouvin
FF: Fabrizio Furano
PS: Philippe Seraphin
DB: David Bouvet
LD: Laurent Duflot

## Introduction by FC:
 
This doc is a discussion about the future from the sites p.o.v. collecting their input about their vision of storage, DPM in particular, at the HC-LHC scale and in the context of DOMA.
We agree that we are ok for the next 5y with the new stuck which is being deployed at sites. But we need a vision of the longer times investments.

OK: WLCG Ops are writing a similar document collecting their concerns and vision of the DPM project. They asked the developers also to put some technical remarks. In summary: evolutions to cope with DOMA directives are likely to be possible but the project was not clearly mandated for this.

WLCG operations are writing a similar document expecting to be a companion document of the DPM WP and addressing some aspects as  sustainability of DPM for the long term. They asked the developers also to put some technical remarks. For example: evolutions to cope with DOMA directives are likely to be possible but the project was not clearly mandated for this.

DPM WP should be companion/complementary document, focusing on the feedback from the community.

The document should be reasonably short and a ideal deadline should be the beginning of November.
The CERN IT makes a annual projects review by the beginning of November and (thought there is no guarantee that DPM will be discussed) this is a good opportunity for the DPM community voice to be heard. The WLCG Ops document will also be ready by that date.


## Round the table:

AD: which form should be given to the doc?

FC AS: Different chapter with topics (already listed in the doc). Each site/region contribute separately along the lines of these chapters. All this in a google doc

https://docs.google.com/document/d/1pcKJhaz6MEnF8VQMH3oeeSsCBJSiIdH7QzW7smf4s8Q/edit#

The different contributions will then be merged.

FC: various links have been added to the end of the google doc.  We can reuse any piece of info and get inputs from the answers to Storage part of the site survey, Data-Lake straw-man model, CHEP presentations etc.

AD: INFN sites will report about INFN discussions on data lakes project. All sites may report something about their funding agency strategy concerning storage.

BG: EGI will contribute, of course with a slightly different p.o.v.

PV :
1/ outputs of DOMA activities are unknown for now - some requirements can be critical for storage to be able to participate (e.g. WLCG/SciToken support) other may be less critical for smaller sites (QoS?) 2/ No motivation to migrate to the different storage technology at least till HL-LHC and then reevaluate how DPM features fits DOMA requirements for site of our size
3/ performance / scalaibility limits - legacy DPM / SRM had hard time to serve our ~ 5k cores, DPM DOME performance at least an order of magnitude better - fine for our site for a long time

AS: This is a good point for all sites to contribute: what is their attitude toward storage tech migration?

MK RS TK : Tokyo is ready to contribute based on their experience. It is the only site in JP and the biggest DPM site. This site can thus provide a very important and particular p.o.v. to the discussion.

LD: Discussions from DOMA-QoS. Should DPM support QoS?
Also most sites have Raid-6 storage and one way to increase capacity is to pass to a JBOD setup. This requires resilience to disk loss and effective tools to identify the portion of data lost and to notify it to central operations. Which would be the adaptability of DPM to this type of setup.
From DOMA-Access: also the cache setup is a possible solution, which are the prospectives of cache functionalities in DPM

OK: One key point is that we do not yet know what DOMA will ask from us in the end. We may write a list of things which are "in the air". But it is not worth to enter in the details. The main point is to raise (and try to answer) questions like
  - which is the sustainability of the project to HL-LHC scale? is there a strong community in place? a prospective for a strong collaborations? which may be the expected contributors?
- Besides give the input with their current situation and plans, sites should list their doubts/questions.

MJ : got some experience from writing the HSF document. It is better to start from a core of people then to circulate the document to a broader community. 2 weeks should be enough to start  the process of writing. It is necessary to send a recall in a week.


## Items in the document (FC):

- Intro: goal of the document
- DPM as infrastructure: summarize the infra correctly deployed, how much storage, how many sites, ...
- DPM as storage tech, current status: do you agree that with the new stack we will be fine for the next 5y (in case put remarks)? Not worth to mention legacy stack and the migration process. See how everyone feels about the storage
- Ongoing R&D activities
- Ops, RH, investments: which are your investment plans for the storage (manpower, HW, ...) and how they impact or are impacted by the choice of storage and the evolution of DPM.
- Needs ... : this is your long time vision. Innovative suggestions can help to attract people and projects.

everybody agrees with this items. More may be added afterward if the need shows up.

## How to proceed/Action items:

- prepare a skeleton (FC & AS) with the items and a short explication for each one. By the beginning of next week.
- everybody is warned by mail and people should start to contribute. 2 weeks deadline.
- FC & AS will send reminds and schedule a new meeting in 2 weeks.
- At the meeting the contributions will be  discussed
- Doc redaction to integrate the different contributions (may take 1 week)
- Once a first draft is ready send it to the DPM forum asking for comment (with 1/2 weeks deadline)

 

09:00 → 09:20 Introduction

The idea of writing a DPM white paper comes out of the discussion "Looking into the future" organized during the last DPM workshop 2019 in Bern.

The intent is to gather inputs from sites related to medium and long term future of DPM, share views towards the possible evolutions of our DPM infrastructures in the context of WLCG DOMA and for the HL-LHC era.

The first objective is to make sure a quorum of sites/countries/organizations or VOs are ready to contribute to such document.

09:20 → 09:40 Discussion

Discussion : process, dead-line, scope and contents of this DPM White paper

Link to Google doc