Storage Interfaces WG Meeting
Agenda:
https://indico.cern.ch/conferenceDisplay.py?confId=233970

Interfaces Intro (Wahid)

Philippe queried whether gFal2 would be well enough supported to be able to handle all that is assigned to it here (e.g. for deletion it will need to speak to all storage types). 
Oliver said yes it would; noted that removing srm may make it simpler in longer term; as would not having to support gFal1 and other legacy stuff. 

Dirk pointed out on the subject of Clouds / S3 that CERN had tested both a Huawei Storage Appliance and Open Stack and that these could be made available for further tests.

Ricardo/Oliver mentioned that DPM supported S3 backends. 
Michel pointed out that this using it with DPM wasn't then "pure" cloud storage. 
Dirk noted that as long as we have our current security / authentication requirements then we have to have something additional to S3.

Simone asked to clarify whether with gFal2 the error messages from deletion on different storage backends are consistent (or it passes through the particular storage message )
It was confirmed that each error is mapped to the same final one. 

Actions (from inside talk) 
ATLAS test gFal2 functionality for their use-cases (deletion / service discovery) : Simone / Wahid
ATLAS full non-SRM site demonstrator: Wahid / Simone 
Namespace instead of space-token demonstrator : Wahid /DPM /Simone

Noone objected to the proposed merger of groups and widening of remit to follow up on all post-TEG recommendations (inc. cloud storage and data-access protocols) 
Action:   Wahid will circulate a revised mandate.

ATLAS WebDAV plans (Cedric)

Cedric asked if StoRM has an interface yet. It was confirmed that it does. 
Ricardo asked if DPM should do an urgent release for the fix to the issue raised in the talk. Cedric confirmed it was OK to wait a month for a release with other things in it. 
Dirk said there EOS is in contact with the DPM/Davix team and expects to reuse their developments but there is no concrete schedule for WebDAV on EOS as of now. Fabrizio said that in the next week or two it should be possible to confirm a more concrete date for xrootd webdav plugin. 

It was asked why ATLAS is using requests - it is for the integration with the pure python framework in rucio.
Philippe asked why not use gFal2 and Cedric said it was easy to do in requests. 
Philippe then asked so why does gFal2 support Dav - Oliver replied it was needed for FTS anyway. 

Time taken for the renaming: Cedric said it was ~ 20 Hz and so a big T2 in a few days. 
Is the LFC renaming a bottleneck: Not tested.

Wahid noted that it seemed a lot of effort for a one-off renaming and it was confirmed that ATLAS saw this as the first of potentially more use-cases including the successor to "dq2-get" (user-download) .
 A discussion followed of the effort involved and there was a feeling (e.g. from Michel and Simone) that the deployment should be less effort that for xrootd (and accompanying Atlas name2name plugin). 
Ricardo noted with Dav you also get some tools for free.

Next steps:  wlcg perspective have them as real services (both xrootd and Webdav) for SAM Tests 
Michel - is there not in gocdb? Yes xrootd exists as a service tag - for webdav not sure it exists. 
ops task force for federations - Dominico could take forward 
 
Ricardo noted it is being published in BDII for DPM .
But Simone said for Sam tests you need to have it in goc.

FTS and pure-GRID FTP (Alejandro)

Wahid queried whether the xrootd and WebDAV will completely offer the complete transfers without SRM - and it was confirmed that it would.
Wahid also offered some testing in the UK. 

Disk-usage and quota on namespace 

Wahid suggested that we draw up a concrete proposal on an alternative to spacetokens reporting usage (and potentially applying quotas) on the second directory-level under the vo name … e.g. atlas/atlasdatadisk 

Philippe objected to the loss of orthogonality between Space Token and namespace and suggested LHCb would prefer the endpoint as currently provided by castor with a URL ending in ?serviceclass=
A discussion followed. Some of the points raised include:
Lustre and gpfs provide quotas by group / file set respectively.
An analogy was made to the partition table and the "df" command.
Some sites have been asked also to protect one VO from another's usage (so not just spaces within the VO that need to be guaranteed).

No definite structure was agreed. Instead it was suggested for next meeting we should collect info on current quota implementations and iterate on a "wish-list". 
Action: Wahid to circulate proto-proposal based on what was discussed. 

Data access protocols - WLCG direction (Wahid). 

Markus mentioned that xrootd was becoming defacto core protocol. (As CMS had requested it at all sites and ATLAS were also starting to do so). 
Brian said we should not be tied to one thing but also be clear how to migrate. 
It was mentioned that the point presented that you could have a different protocol that was actually used in addition to core protocol would allow for such transitions. 

Markus and Dirk questioned the suggestion that Fuse could be part of a solution here. Wahid and others clarified that the idea was more fuse in conjunction with gFal2 (or similar). 
Though that is just an idea - and there was no particular appetite for it expressed in the meeting.

On rfio retirement: 
Michel thought it was important for experiments to state if they care for rfio to be retired. 
Brian said that CMS would be happy for it to go. 
Markus mentioned that also from the development side there was a need to move away from rfio.

Noone objected to rfio retirement - though clearly it would take some time as most if not all WLCG DPM sites use it currently in production. 
Action: Wahid to circulate plan / timeline - for moving to ops team.