DOMA / ACCESS Meeting

Europe/Zurich
513/1-024 (CERN)

513/1-024

CERN

50
Show room on map
Frank Wuerthwein (Univ. of California San Diego (US)), Ilija Vukotic (University of Chicago (US)), Markus Schulz (CERN), Stephane Jezequel (LAPP-Annecy CNRS/USMB (FR)), Xavier Espinal (CERN)

Attendance: 
Frank Wuerthwein, Ilija Vukotic, Markus Schulz, Stephane Jezequel, Xavier Espinal, Nikolai Hartmann, Andrew Hanushevsky, Diego Ciangottini, Gonzalo Merino, Horst Severini, Johannes Elmsheuser, Jose Flix Molina, Oxana Smirnova, Riccardo Di Maria, Tigran Mkrtchyan, David Smith

Announcement: 
LHCOPN/LHCONE workshop, Jan 13th-14th 2020, 31/3-004 - IT Amphitheatre (CERN) - https://indico.cern.ch/event/828520/

A general DOMA presentation is scheduled for 30 minutes, aiming to cover all aspects (TPC, ACCESS).

Nikolai Hartmann presentation:

See slides attached.

Update on cache tests at LMU Munich using tens of TB pileup samples.

Successful running of XCache in ATLAS production environment.

Running XCache with individual disks beneficial (compared to RAID6): significantly reduces load and wait times; peak I/O also increased for parallel disk reads/writes.

Next plans:
- continue stress tests by removing I/O limit on XCache queue and running all jobs through XCache;
- XCache cluster;
- implement checksum test for fully cached files;
- continue tests with analysis jobs;
- test remote processing (currently reading from a neighbouring site).

General comment: there will be blockwise checksums.

 

Frank Wuerthwein presentation:

See slides attached.

Towards a Data Lake for the HL-LHC Era: recap of previous presentations.

Proposal for a “Hierarchical Storage”:
- keep most data in “active archive” on cheap, and high latency media;
- keep a “golden copy” on redundant high availability disk;
- Regional Caches at processing centres, where the size of the region is determined by latency tolerance of
application;
- this will potentially lead to x4 less disk space for better availability of data. 

Description of SoCal prototype.

Most likely reuse of files in cache is zero (R&D being pursued by Diego Ciangottini, Daniele Spiga, et al.).

For most files, less than 20% of file is read (R&D not presently done by anybody).

To follow up (after having clarified on CMS NanoAOD format).

There are minutes attached to this event. Show them.
    • 17:30 17:35
      Introduction 5m
      Speakers: Frank Wuerthwein (UCSD), Frank Wuerthwein (Univ. of California San Diego (US)), Ilija Vukotic (University of Chicago (US)), Stephane Jezequel (LAPP-Annecy CNRS/USMB (FR))
    • 17:35 18:00
      Update on Xcache (TBC) 25m
      Speaker: Nikolai Hartmann (Ludwig Maximilians Universitat (DE))
    • 18:00 18:20
      Towards a Data Lake for the HL-LHC Era 20m
      Speakers: Frank Wuerthwein (Univ. of California San Diego (US)), Frank Wuerthwein (UCSD)
    • 18:20 18:25
      AOB 5m