eulake technical coordination

Tuesday 10 Apr 2018, 16:00 → 17:00 Europe/Zurich

4th eulake technical coordination meeting

Present: Wybrand and Jean-Marie (SARA), Andrew (NIKHEF), Crystal (Aarnet), Enrico (CERN-IT/ST) Simone and Xavi (CERN-IT/LCG)

Apologies: Brian (STFC)

Round table:

CERN

• 5 sites configured: Sara, Dubna, RAL, CERN-Wigner and CERN-Meyrin
• Basic datalakes features tested:
  • "scattered" replicas
  • "striping" across sites
  • File layout automatic transition based on a defined time, ie: sys.lru.convert.match="*:1h"
    • From 2 replicas to 1, or from 2 replicas to 4+2, etc.
  • Simulated site failure on striping scenarios

AARNET:

• One FST is ready in Melbourne with three disk endpoints
• Two more FST will come in the following weeks in different locations: Brisbane and Perth
• No major issues setting up the endpoints, felt largely easier than fully compliant EOS instance.

NIKHEF

• EOS software being installed in two Virtual Machines and expect to be ready in the following weeks
• Need more information on node configuration parameters and firewall needs. Action on Enrico to provide it.

SARA

• Current FST running smoothly since 5 weeks
• Small dcache installation ready with newest software (4.1). The volume will be exposed via NFS share.
  • Possibility to plug HSM on this endpoint
• Suggestion to keep a list of datalake site subnets for configuration. Besides the connectivity required to the central management node at CERN, the copies between sites need to be granted via the firewall. Always same port (1095). Action on Xavi to evaluate the best way to provide this.

AOB:

Simone provided a quick overview and summary of the common data management and data lake sessions during last week's WLCG and HSF workshop (https://indico.cern.ch/event/658060/). The main outcome is the need to have a common project in the context of WLCG embracing data lakes and storage consolidation, data transfer mechanism, protocols, storage interoperability, caching technologies and content delivery. This proposal will be presented and discussed at the Grid Deployment Board tomorrow (https://indico.cern.ch/event/651352/)

Next meeting: 24th of April

 

16:00 → 16:10 News and Announcements

Speaker: Xavier Espinal (CERN)

Joint WLCG and HSF workshop: https://indico.cern.ch/event/658060/timetable/

• Common Data Management and Data Lakes session:
  • https://indico.cern.ch/event/658060/sessions/266380/#20180327

16:10 → 16:30 Round table

eulake-coordination-09042018-filetransition.pdf

CERN:

- 5 sites configured: Sara, Dubna, RAL, CERN-Wigner and CERN-Meyrin

- Basic datalakes features tested:

• "scattered" replicas
• "striping" across sites
• File layout automatic transition based on a defined time, ie: sys.lru.convert.match="*:1h"
  • From 2 replicas to 1, or from 2 replicas to 4+2, etc.
• Simulated site failure on striping scenarios

16:30 → 16:35 Wrap-up and next meeting