24th International Conference on Computing in High Energy & Nuclear Physics

Australia/Adelaide
Adelaide Convention Centre

Adelaide Convention Centre

Adelaide, Australia
Paul Douglas Jackson (University of Adelaide), Waseem Kamleh (University of Adelaide)
Description

Adelaide

Welcome to the 24th International Conference on Computing in High-Energy and Nuclear Physics. The CHEP conference series addresses the computing, networking and software issues for the world’s leading data‐intensive science experiments that currently analyse hundreds of petabytes of data using worldwide computing resources.

2KWAdelaide Convention Centre

CHEP 2019 will be held in Adelaide, South Australia, between Monday-Friday 4-8 November 2019. The venue for the CHEP 2019 conference is the Adelaide Convention Centre, conveniently located on the picturesque banks of the Torrens Lake in the heart of the city.

The optional pre-conference WLCG & HSF workshop will be held at the North Terrace campus of the University of Adelaide between Saturday-Sunday 2-3 November 2019.

More details about the conference can be found at the conference website: CHEP2019.org

 

Participants
  • Abhishek Sharma
  • Adam Davis
  • Adam Leinweber
  • Adam Virgili
  • Adam Wegrzynek
  • ADEYEMI JOHN ADEWALE
  • Aivaras Silale
  • Albert Kong
  • Alberto Di Meglio
  • Alec Hannaford Gunn
  • Alessandra Forti
  • Alessandro Di Girolamo
  • Alex Sim
  • Alex Westin
  • Alexander Adler
  • Alexander Held
  • Alexey Anisenkov
  • Amber Boehnlein
  • Andre Sailer
  • Andre Scaffidi
  • Andrea Bocci
  • Andrea Ceccanti
  • Andrea Dell'Acqua
  • Andrea Formica
  • Andrea Rizzi
  • Andrea Sciabà
  • Andrea Valassi
  • Andreas Heiss
  • Andreas Joachim Peters
  • Andreas Petzold
  • Andreas Wicenec
  • Andrei Dumitru
  • Andrei Gheata
  • Andrew Bohdan Hanushevsky
  • Andrew Davis
  • Andrew Lahiff
  • Andrew Melo
  • Andrey Baginyan
  • Anna Mullin
  • Anna Woodard
  • Anthony Thomas
  • Anthony Williams
  • Antonio Boveia
  • Antonio Delgado Peris
  • Antonio Perez-Calero Yzquierdo
  • Antonín Květoň
  • Arantza De Oyanguren Campos
  • Arturo Sanchez Pineda
  • Atsushi Mizukami
  • Attila Krasznahorkay
  • Barbara Martelli
  • Bastien Gounon
  • Batool Safarzadeh Samani
  • Beijiang Liu
  • Ben Couturier
  • Ben Galewsky
  • Ben Menadue
  • BENDA XU
  • Benedikt Riedel
  • Benedikt Volkel
  • Benjamin Krikler
  • Benjamin LaRoque
  • Benjamin Roberts
  • Bjarte Kileng
  • Borut Paul Kersevan
  • Bradley Wegener
  • Brian Bockelman
  • Brij Kishor Jashal
  • Bruno Hoeft
  • Carl Vuosalo
  • Carlos Fernando Gamboa
  • Carsten Daniel Burgard
  • Caterina Doglioni
  • Caterina Marcon
  • Catherine Biscarat
  • Charles Michael Grant
  • CHARLIN MISAMBU LUPAMANA
  • Chiara Ilaria Rovelli
  • Chris Burr
  • Chris Pinkenburg
  • Christian Kahra
  • Christian Schmitt
  • Christian Voss
  • Christoph Wissing
  • Christophe Haen
  • Christopher Jones
  • Chun-Yu Lin
  • Chunhua Li
  • Concezio Bozzi
  • Cornelius Grunwald
  • Curtis Abell
  • Dagmar Adamova
  • Damir Duvnjak
  • Daniel Crawford
  • Daniel Hugo Cámpora Pérez
  • Daniel Peter Traynor
  • Daniele Gregori
  • Daniele Spiga
  • Danila Oleynik
  • Danilo Piparo
  • Dario Barberis
  • David Britton
  • David Cameron
  • David Casper
  • David Crooks
  • David Dossett
  • David Kelsey
  • David Lange
  • David Lawrence
  • David Michael South
  • David Rohr
  • David Schultz
  • Dennis Noll
  • Derek Leinweber
  • Diana Scannicchio
  • Diego Rodríguez Rodríguez
  • Dirk Pleiter
  • Dmitriy Maximov
  • Dominik Muller
  • Doris Kim
  • Dorothea vom Bruch
  • Doug Benjamin
  • Eduardo Rodrigues
  • Edward Karavakis
  • Elizabeth Gallas
  • Elizabeth Sexton-Kennedy
  • Emilio Meschi
  • Emily Filmer
  • Emma Torro Pastor
  • Enric Tejedor Saavedra
  • Enrico Bocchi
  • Enrico Gamberini
  • Eric Cano
  • Eric Vaandering
  • Ethan Carragher
  • Evgeny Lavrik
  • Fabio Alberto Espinosa Burbano
  • Fabio Hernandez
  • Farid Ould-Saada
  • Fazhi QI
  • Federica Legger
  • Federico Carminati
  • Federico Stagni
  • Fedor Ratnikov
  • Felice Pantaleo
  • Fernando Harald Barreiro Megino
  • Filippo Costa
  • Flavio Pisani
  • Frank Berghaus
  • Frank Gaede
  • Frank Winklmeier
  • Frederic Deliot
  • Gagik Gavalian
  • Gang Chen
  • Gang LI
  • Gavin Kennedy
  • Gene Van Buren
  • Gerardo Ganis
  • Ghita Rahal
  • Gianfranco Sciacca
  • Gianluca Peco
  • Gilles Grasseau
  • Giulia Tuci
  • Giulio Eulisse
  • Giuseppe Andronico
  • Giuseppe Cerati
  • Go Iwai
  • Gordon Watts
  • Graeme A Stewart
  • Greg Corbett
  • Grigoriy RYBKIN
  • Grzegorz Jereczek
  • Guang Zhao
  • Gvozden Nešković
  • Hannah Short
  • Hannes Sakulin
  • Haykuhi Musheghyan
  • Heather Gray
  • Heidi Marie Schellman
  • Henry Fredrick Schreiner
  • Horst Severini
  • Hristo Umaru Mohamed
  • Huaimin Liu
  • Hugo Gonzalez Labrador
  • Ian Bird
  • Ian Collier
  • Ian Fisk
  • Igor Sfiligoi
  • Igor Soloviev
  • Ilya Komarov
  • Imran Latif
  • Ioana Ifrim
  • Iouri Smirnov
  • iris sabo napadensky
  • Ivan Kisel
  • Ivana Hrivnacova
  • Jacinda Ginges
  • Jakob Blomer
  • Jakub Moscicki
  • James Biddle
  • James Collinson
  • James Kahn
  • James Letts
  • James Zanotti
  • jan balewski
  • Jan de Cuveland
  • Jarryd Day
  • Jason Oliver
  • Jason Webb
  • Jean-Roch Vlimant
  • Jeff Adie
  • Jeff Porter
  • Jeffrey LeFevre
  • Jennifer Ngadiuba
  • Jerome LAURET
  • Jerome Odier
  • Jiahui Wei
  • Jim Pivarski
  • Jiri Chudoba
  • Joel Closier
  • Joern Adamczewski-Musch
  • Johannes Elmsheuser
  • Johnny Raine
  • Jonas Eschle
  • Jorn Schumacher
  • Jose Flix Molina
  • Jose Hernandez
  • Josh Charvetto
  • Joshua Heneage Dawes
  • Juan Chen
  • Julia Andreeva
  • Julien Leduc
  • Julius Hrivnac
  • Junichi Tanaka
  • Justin Albury
  • Kadir Utku Can
  • Karl Ehatäht
  • Katy Ellis
  • Kejun Zhu
  • Kenneth Richard Herner
  • Kenyi Hurtado Anampa
  • Kevin Greif
  • Kevin Lannon
  • Kevin Pedro
  • Kevin Varvell
  • Kihyeon Cho
  • Kim Smith
  • Kinga Anna Wozniak
  • Koji Hara
  • Lara Lloret Iglesias
  • Latchezar Betev
  • laura perini
  • Laura Sargsyan
  • Leo Piilonen
  • Leonid Serkin
  • Liam Hockley
  • Linghui Wu
  • Lirim Osmani
  • Liviu Valsan
  • Lloyd Hollenberg
  • Lorenzo Moneta
  • Lu Wang
  • Luca dell'Agnello
  • Luca Mascetti
  • Lucia Silvestris
  • Luis Fernandez Alvarez
  • Luis Granado Cardoso
  • Lukas Alexander Heinrich
  • Lukas Arnold
  • Lukas Layer
  • Lukasz Kamil Graczykowski
  • Lyn Beazley
  • Maciej Gladki
  • Maiken Pedersen
  • Manfred Peter Fackeldey
  • Mansi Kasliwal
  • Manuel Giffels
  • Manuel Rodriguez
  • Marc Dobson
  • Marcel Rieger
  • Marcin Nowak
  • Marcin Slodkowski
  • Marco Cattaneo
  • Marco Clemencic
  • Marco Milesi
  • Marco Zanetti
  • Maria Alandes Pradillo
  • Maria Girone
  • Marilena Bandieramonte
  • Mario Lassnig
  • Marion Devouassoux
  • Mark Gray
  • Mark Neubauer
  • Markus Klute
  • markus schulz
  • Marten Ole Schmidt
  • Marten Teitsma
  • Martin Adam
  • Martin Gasthuber
  • Martin Ritter
  • Martin Sevior
  • Martin Soderen
  • Martin White
  • Marzena Lapka
  • Masahiko Saito
  • Mason Proffitt
  • Matevz Tadel
  • Matt Westlake
  • Matthew Feickert
  • Matthew Nguyen
  • Matthias Komm
  • Matthias Richter
  • Maurizio Pierini
  • Max Fischer
  • Maxim Gonchar
  • Maxim Potekhin
  • Maxim Storetvedt
  • Maxim Voronkov
  • Michael Hildreth
  • Michael Lettrich
  • Michael Papenbrock
  • Michael Poat
  • Michael Schuh
  • Michal Svatos
  • Michel Jouvin
  • Michel Villanueva
  • Michiru Kaneda
  • Miha Muskinja
  • Mihaela Gheata
  • Milos LOKAJICEK
  • Ming Tang
  • Minh Huynh
  • Minh Tan Ha
  • Mircho Rodozov
  • Mischa Batelaan
  • Mohammad Al-Turany
  • Natalya Melnikova
  • Nicholas Leerdam
  • Nick Smith
  • Niklas Nolte
  • Nobuo Sato
  • Norman Graf
  • Ofer Rind
  • Oksana Shadura
  • Oleg Samoylov
  • Oliver Gutsche
  • Oliver Sander
  • Olof Barring
  • Othmane Bouhali
  • Oxana Smirnova
  • Pablo Saiz
  • Padric McGee
  • Pamfilos Fokianos
  • Paolo Calafiura
  • Pat Scott
  • Patrick Fuhrmann
  • Paul Gessinger
  • Paul Jackson
  • Paul James Laycock
  • Paul Lasky
  • Paul Nilsson
  • Pawel Grzywaczewski
  • Pedro Andrade
  • Penelope Constanta
  • Petar Rados
  • Peter Elmer
  • Peter Hristov
  • Peter Kodys
  • Peter Love
  • Phiala Shanahan
  • Philip Grace
  • Piero Vicini
  • Piotr Konopka
  • Placido Fernandez Declara
  • Qingmin Zhang
  • Qiulan Huang
  • Qiumei MA
  • Rafal Dominik Krawczyk
  • Randall Sobie
  • Raul Jimenez Estupinan
  • Reda Tafirout
  • Remi Ete
  • Remi Mommsen
  • Rene Caspart
  • Revital Kopeliansky
  • Ricardo Brito da Rocha
  • Riccardo Farinelli
  • Richard Cziva
  • Richard Dubois
  • Richard Hughes-Jones
  • Riley Patrick
  • Robert Andrew Currie
  • Robert Bainbridge
  • Robert William Gardner Jr
  • Rodrigo Sierra
  • Roel Aaij
  • Roger Jones
  • Rohini Joshi
  • Roland Sipos
  • Romain Wartel
  • Roman Dzhygadlo
  • Rosario Esposito
  • Rose Smail
  • Rosen Matev
  • Ross Young
  • Ruben Domingo Gaspar Aparicio
  • Ruben Shahoyan
  • Ruth Weiss Babai
  • Ryan Bignell
  • Ryan Taylor
  • Ryosuke Itoh
  • Ryu Sawada
  • Sabrina Amrouche
  • Sabrina Einecke
  • Sally Robertson
  • Sandro Wenzel
  • Sang Un Ahn
  • Sanjay Bloor
  • Santiago Gonzalez De La Hoz
  • Scott Snyder
  • Sebastian Bukowiec
  • Sebastian Lopienski
  • SEBASTIEN GADRAT
  • Sergey Gorbunov
  • Sergey Linev
  • Shanette De La Motte
  • Sharon Johnson
  • Shawn McKee
  • Sheng Sen Sun
  • Shigeki Misawa
  • Shiyuan Fu
  • Shuopin Wen
  • Siarhei Padolski
  • Silvana Santucci
  • Silvio Pardi
  • Simon Blyth
  • Simon Lee
  • Simon Voigt Nesbo
  • Simone Campana
  • Sitong An
  • Slava Kitaeff
  • Sofija Antic
  • Stefan Roiser
  • Stefan Wunsch
  • Stefano Bagnasco
  • Stefano Dal Pra
  • Stefano Giagu
  • Stefano Petrucci
  • Stefano Piano
  • Stefano Spataro
  • STELLA CHRISTODOULAKI
  • Stephan Hageboeck
  • Stephane Jezequel
  • Stephen Ord
  • Steven Farrell
  • Steven Goldfarb
  • Steven Schramm
  • Tadashi Murakami
  • Tadeas Bilka
  • Tamas Bato
  • Teng Jian Khoo
  • Teo Mrnjavac
  • Theo Motta
  • Thiago Rafael Tomei Fernandez
  • Thomas Beermann
  • Thomas Britton
  • Thomas Hartmann
  • Thomas James
  • Thomas Kuhr
  • Tibor Šimko
  • Tigran Mkrtchyan
  • Tim Smith
  • Tomas Howson
  • Tomas Lindén
  • Tommaso Boccali
  • Tomoaki Nakamura
  • Tomoe Kishimoto
  • Tony Cass
  • Toru SUGITATE
  • Tristan Ruggeri
  • Tristan Sudholz
  • Ugo Varetto
  • Ursula Laa
  • Valdas Rapsevicius
  • Vardan Gyurjyan
  • Venkitesh Ayyar
  • Violet Harvey
  • Vladimir Loncar
  • Volker Friese
  • Wahid Bhimji
  • Wally Melnitchouk
  • Walter Lampl
  • Waseem Kamleh
  • wasif husain
  • Wassef Karimeh
  • wei su
  • Wei Yang
  • Weidong Li
  • Wenhao Huang
  • William Detmold
  • William Panduro Vazquez
  • Wuming Luo
  • Xavier Espinal
  • Xiaobin Ji
  • Xiaomei Zhang
  • Xiaowei Jiang
  • Xin Zhao
  • Xinchou Lou
  • Xingtao Huang
  • Yan Huang
  • YANG WANG
  • Yang Zhang
  • Yanjia Xiao
  • Yao Zhang
  • Yaodong Cheng
  • Ye Yuan
  • Yu Nakahama Higuchi
  • Yujiang Bi
  • Zeno Kordov
  • Zhang Aiqiang
  • zhen Huang
  • Zhenjing CHENG
  • Zhiming Li
  • Ziheng Chen
  • Ziyan Deng
    • 08:00 09:00
      Registration 1h Foyer F

      Foyer F

      Adelaide Convention Centre

    • 09:00 10:30
      Plenary: Welcome / DUNE / Belle II Hall G

      Hall G

      Adelaide Convention Centre

      Convener: Simone Campana (CERN)
    • 10:30 11:00
      Break 30m
    • 11:00 12:30
      Track 1 – Online and Real-time Computing: Data acquisition (DAQ) Riverbank R5

      Riverbank R5

      Adelaide Convention Centre

      Convener: Chunhua Li
      • 11:00
        A novel centralized slow control and board management solution for ATCA blades based on the Zynq Ultrascale+ System-on-Chip 15m

        Data acquisition systems (DAQ) for high energy physics experiments utilize complex FPGAs to handle unprecedented high data rates. This is especially true in the first stages of the processing chain. Developing and commissioning these systems becomes more complex as additional processing intelligence is placed closer to the detector, in a distributed way directly on the ATCA blades, in the other hand, sophisticated slow control is as well desirable. In this contribution, we introduce a novel solution for ATCA based systems, which combines the IPMI, a Linux based slow-control software, and an FPGA for custom slow-control tasks in one single Zynq Ultrascale+ (US+) System-on-Chip (SoC) module.

        The Zynq US+ SoC provides FPGA logic, high-performance ARM-A53 multi-core processors and two ARM-R5 real-time capable processors. The ARM-R5 cores are used to implement the IPMI/IPMC functionality and communicate via backplane with the shelf manager at power-up. The ARM-R5 are also connected to the power supply (via PMBus), to voltage and current monitors, to clock generators and jitter cleaners (via I2C, SPI). Once full power is enabled from the crate, a Linux based operating system starts on the ARM-A53 cores. The FPGA is used to implement some of the low-level interfaces, including IPBus, or glue-logic. The SoC is the central entry point to the main FPGAs on the motherboard via IPMB and TCP/IP based network interfaces. The communication between the Zynq US+ SoC and the main FPGAs uses the AXI chip-to-chip protocol via MGT pairs keeping infrastructure requirements in the main FPGAs to a minimum.

        Speaker: Oliver Sander (KIT - Karlsruhe Institute of Technology (DE))
      • 11:15
        DAQling: an open source data acquisition framework 15m

        The data acquisition (DAQ) software for most applications in high energy physics is composed of common building blocks, such as a networking layer, plug-in loading, configuration, and process management. These are often re-invented and developed from scratch for each project or experiment around specific needs. In some cases, time and available resources can be limited and make development requirements difficult or impossible to meet.
        Moved by these premises, our team developed an open-source lightweight C++ software framework called DAQling, to be used as the core for the DAQ systems of small and medium-sized experiments and collaborations.
        The framework offers a complete DAQ ecosystem, including communication layer based on the widespread ZeroMQ messaging library, configuration management based on the JSON format, control of distributed applications, extendable operational monitoring with web-based visualization, and a set of generic utilities. The framework comes with minimal dependencies, and provides automated host and build environment setup based on the Ansible automation tool. Finally, the end-user code is wrapped in so-called “Modules”, that can be loaded at configuration time, and implement specific roles.
        Several collaborations already chose DAQling as the core for their DAQ systems, such as FASER, RD51, and NA61. We will present the framework and project-specific implementations and experiences.

        Speaker: Enrico Gamberini (CERN)
      • 11:30
        DUNE DAQ R&D integration in ProtoDUNE Single-Phase at CERN 15m

        The DAQ system of ProtoDUNE-SP successfully proved its design principles and met the requirements of the beam run of 2018. The technical design of the DAQ system for the DUNE experiment has major differences compared to the prototype due to different requirements and the environment. The single-phase prototype in CERN is the major integration facility for R&D aspects of the DUNE DAQ system. This covers the exploration of additional data processing capabilities and optimization of the FELIX system, which is the chosen TPC readout solution for the DUNE Single Phase supermodules. One of the fundamental differences is that DUNE DAQ relies on self-triggering. Therefore real-time processing of the data stream for hit and trigger primitive finding is essential for the requirement of continuous readout, where Intel AVX register instructions are used for better performance. The supernova burst trigger requires a large and fast buffering technique, where 3D XPoint persistent memory solutions are evaluated and integrated. In order to maximize resource utilization of the FELIX hosting servers, the elimination of the 100Gb network communication stack is desired. This implies the design and development of a single-host application layer, which is a fundamental element of the self-triggering chain.
        This paper discusses the evaluation and integration of these developments for the DUNE DAQ, in the ProtoDUNE environment.

        Speaker: Roland Sipos (CERN)
      • 11:45
        FELIX: commissioning the new detector interface for the ATLAS trigger and readout system 15m

        After the current LHC shutdown (2019-2021), the ATLAS experiment will be required to operate in an increasingly harsh collision environment. To maintain physics performance, the ATLAS experiment will undergo a series of upgrades during the shutdown. A key goal of this upgrade is to improve the capacity and flexibility of the detector readout system. To this end, the Front-End Link eXchange (FELIX) system has been developed. FELIX acts as the interface between the data acquisition; detector control and TTC (Timing, Trigger and Control) systems; and new or updated trigger and detector front-end electronics. The system functions as a router between custom serial links from front end ASICs and FPGAs to data collection and processing components via a commodity switched network. The serial links may aggregate many slower links or be a single high bandwidth link. FELIX also forwards the LHC bunch-crossing clock, fixed latency trigger accepts and resets received from the TTC system to front-end electronics. FELIX uses commodity server technology in combination with FPGA-based PCIe I/O cards. FELIX servers run a software routing platform serving data to network clients. Commodity servers connected to FELIX systems via the same network run the new multi-threaded Software Readout Driver (SW ROD) infrastructure for event fragment building, buffering and detector-specific processing to facilitate online selection. This presentation will cover the design of FELIX, the SW ROD, and the results of the installation and commissioning activities for the full system in summer 2019.

        Speaker: William Panduro Vazquez (Royal Holloway, University of London)
      • 12:00
        Integration of custom DAQ Electronics in a SCADA Framework 15m

        LHCb is one of the 4 experiments at the LHC accelerator at CERN. During the upgrade phase of the experiment, several new electronic boards and Front End chips that perform the data acquisition for the experiment will be added by the different sub-detectors. These new devices will be controlled and monitored via a system composed of GigaBit Transceiver (GBT) chips that manage the bi-directional slow control traffic to the Slow Control Adapter(s) (SCA) chips. The SCA chips provide multiple field buses to interface the new electronics devices (I2C, GPIO, etc). These devices will need to be integrated in the Experiment Control System (ECS) that drives LHCb. A set of tools was developed that provide an easy integration of the control and monitoring of the devices in the ECS. A server (GbtServ) provides the low level communication layer with the devices via the several user buses in the SCA chip and exposes an interface for control to the experiment SCADA (WinCC OA), the fwGbt component provides the interface between the SCADA and the GbtServ and the fwHw component, a tool that allows the abstraction of the devices models into the ECS. Using the graphical User Interfaces or XML files describing the structure and registers of the devices it creates the necessary model of the hardware as a data structure in the SCADA. It allows then the control and monitoring of the defined registers using their name, without the need to know the details of the hardware behind. The fwHw tool also provides the facility of defining and applying recipes - named sets of configurations - which can be used to easily configure the hardware according to specific needs.

        Speaker: Luis Granado Cardoso (CERN)
      • 12:15
        Zero-deadtime processing in beta spectroscopy for measurement of the non-zero neutrino mass 15m

        The Project 8 collaboration seeks to measure, or more tightly bound, the mass of the electron antineutrino by applying a novel spectroscopy technique to precision measurement of the tritium beta-decay spectrum. For the current, lab-bench-scale, phase of the project a single digitizer produces 3.2 GB/s of raw data. An onboard FPGA uses digital down conversion to extract three 100 MHz wide (roughly 1.6 keV) frequency regions of interest, and transmits both the time and frequency domain representation of each region over a local network connection for a total of six streams, each at 200 MB/s. Online processing uses the frequency-domain representation to implement a trigger based on excesses of power within a narrow frequency band and extended in time. When the trigger condition is met, the corresponding time-domain data are saved, reducing the total data volume and rate while allowing for more sophisticated offline event reconstruction. For the next phase of the experiment, the channel count will increase by a factor of sixty. Each channel will receive signals from the full source volume, but at amplitudes below what is detectable. Phase-shifted combinations of all channels will coherently enhance the amplitude of signals from a particular sub-volume to detectable levels, and tiling set of such phase shifts will allow the entire source volume to be scanned for events. We present the online processing system which has successfully been deployed for the current, single-channel, phase. We also present the status and design for a many-channel platform.

        Speaker: Benjamin LaRoque
    • 11:00 12:30
      Track 2 – Offline Computing: ML Reconstruction & PID Riverbank R6

      Riverbank R6

      Adelaide Convention Centre

      Convener: Teng Jian Khoo (Universite de Geneve (CH))
      • 11:00
        A Graph Neural Network Approach for Neutrino Signal Reconstruction from LarTPC Raw Data 15m

        The Deep Underground Neutrino Experiment (DUNE) is an international effort to build the next-generation neutrino observatory to answer fundamental questions about the nature of elementary particles and their role in the universe. Integral to DUNE is the process of reconstruction, where the raw data from Liquid Argon Time Projection Chambers (LArTPC) are transformed into products that can be used for physics analysis. Experimental data is currently obtained from a prototype of DUNE (ProtoDUNE) that is built as a full scale engineering prototype and uses a beam of charged particles, rather than a neutrino beam to test the detector response. The reconstruction software consumes on average 35% of the computational resources in Fermilab. For DUNE it is expected that reconstruction will play a far greater, and computationally more expensive role, as signal activity will be significantly reduced upon deployment of the neutrino beam. Consequently, identifying signals within the raw data will be a much harder task. Alternative approaches to neutrino signal reconstruction must be investigated in anticipation of DUNE. Machine learning approaches for reconstruction are being investigated, but currently, no end-to-end solution exists. As part of an end-to-end reconstruction solution, we propose an approach using Graph Neural Networks (GNN) to identify signals (i.e. hits) within the raw data. In particular, since the raw data of LarTPCs are both spatial and temporal in nature, Graph Spatial-Temporal Networks (GSTNs), capable of capturing dependency relationships among hits, are promising models. Our solution can be deployed for both online (trigger-level) and offline reconstruction. In this work, we describe the methodology of GNNs (and GSTNs in particular) for neutrino signal reconstruction and the preliminary results.

        Speaker: Federico Carminati (CERN)
      • 11:15
        Particle Reconstruction with Graph Networks for irregular detector geometries 15m

        We use Graph Networks to learn representations of irregular detector geometries and perform on it typical tasks such as cluster segmentation or pattern recognition. Thanks to the flexibility and generality of the graph architecture, this kind of network can be applied to detector of arbitrarly geometry, representing the detector elements through a unique detector identification (e.g., physical position) and the readout value and embedding as vertices in a graph. We apply this idea to tasks related to calorimetry and tracking in LHC-like conditions, investigating original graph architectures to optimise performance and memory footprint.

        Speaker: Maurizio Pierini (CERN)
      • 11:30
        Interaction networks for jet characterisation at the LHC 15m

        We study the use of interaction networks to perform tasks related to jet reconstruction. In particular, we consider jet tagging for generic boosted-jet topologies, tagging of large-momentum H$\to$bb decays, and anomalous-jet detection. The achieved performance is compared to state-of-the-art deep learning approaches, based on Convolutional or Recurrent architectures. Unlike these approaches, Interaction Networks allow to reach state-of-the art performance without making assumptions on the underlying data (e.g., detector geometry or resolution, particle ordering criterion, etc.). Given their flexibility, Interaction Networks provide an interesting possibility for deployment-friendly deep learning algorithms for the LHC experiments.

        Speaker: Jean-Roch Vlimant (California Institute of Technology (US))
      • 11:45
        A deep neural network method for analyzing the CMS High Granularity Calorimeter (HGCAL) events 15m

        For the High Luminosity LHC, the CMS collaboration made the ambitious choice of a high granularity design to replace the existing endcap calorimeters. The thousands of particles coming from the multiple interactions create showers in the calorimeters, depositing energy simultaneously in adjacent cells. The data are analog to 3D gray-scale image that should be properly reconstructed.

        In this talk we will investigate how to localize and identify the thousands of showers in such events with a Deep Neural Network model. This problem is well-known in the Vision domain, it belongs to the challenging class: "Object Detection" which is significantly a harder task than “only” an image classification/regression because of the mixed goals : the cluster/pattern identification (cluster type), its localization (bounding box), and the object segmentation (mask) in the scene.

        Our project presents a lot of similarities with the ones treated in Industry but accumulates several technological challenges like the 3D treatment. We will present the Mask R-CNN model which has already proven its efficiency in Industry (for 2D images) and how we extended it to tackle 3D HGCAL data. To conclude we will present the first results of this challenge.

        Speaker: Gilles Grasseau (Centre National de la Recherche Scientifique (FR))
      • 12:00
        GRAAL: A novel package to reconstruct data of triple-GEM detectors 15m

        Micro-Pattern Gas Detectors (MPGDs) are the new frontier in between the gas tracking systems. Among them, the triple Gas Electron Multiplier (triple-GEM) detectors are widely used. In particular, cylindrical triple-GEM (CGEM) detectors can be used as inner tracking devices in high energy physics experiments. In this contribution, a new offline software called GRAAL (Gem Reconstruction And Analysis Library) is presented: digitization, reconstruction, alignment algorithms and analysis of the data collected with APV-25 and TIGER ASICs within GRAAL framework are reported. An innovative cluster reconstruction method based on charge centroid, micro-TPC and their merge is discussed, and the detector performances evaluated experimentally for both planar triple-GEM and CGEM prototypes.

        Speaker: Riccardo Farinelli (Universita e INFN, Ferrara (IT))
      • 12:15
        Particle identification algorithms for the Panda Barrel DIRC 15m

        The innovative Barrel DIRC (Detection of Internally Reflected Cherenkov light) counter will provide hadronic particle identification (PID) in the central region of the PANDA experiment at the new Facility for Antiproton and Ion Research (FAIR), Darmstadt, Germany. This detector is designed to separate charged pions and kaons with at least 3 standard deviations for momenta up to 3.5 GeV/c covering the polar angle range of 22-140 degree.

        An array of microchannel plate photomultiplier tubes is used to detect the location and arrival time of the Cherenkov photons with a position resolution of 2 mm and time precision of about 100 ps. Two reconstruction algorithms have been developed to make optimum use of the observables and to determine the performance of the detector. The "geometrical reconstruction" performs PID by reconstructing the value of the Cherenkov angle and using it in a track-by-track maximum likelihood fit. This method mostly relies on the position of the detected photons in the reconstruction, while the "time imaging" utilizes both, position and time information, and directly performs the maximum likelihood fit using probability density functions determined analytically or from detailed simulations.

        Geant4 simulations and data from the particle beams where used to optimize both algorithms in terms of PID performance and reconstruction speed. We will present current status of development and discuss advantages of each algorithm.

        Speaker: Dr Roman Dzhygadlo (GSI - Helmholtzzentrum fur Schwerionenforschung GmbH (DE))
    • 11:00 12:30
      Track 3 – Middleware and Distributed Computing: Workload Management & Cost Model Riverbank R3

      Riverbank R3

      Adelaide Convention Centre

      Convener: Catherine Biscarat (L2I Toulouse, IN2P3/CNRS (FR))
      • 11:00
        The DIRAC interware: current, upcoming and planned capabilities and technologies 15m

        Efficient access to distributed computing and storage resources is mandatory for the success of current and future High Energy and Nuclear Physics Experiments. DIRAC is an interware to build and operate distributed computing systems. It provides a development framework and a rich set of services for the Workload, Data and Production Management tasks of large scientific communities. A single DIRAC installation provides a complete solution for the distributed computing of one, or more than one collaboration. The DIRAC Workload Management System (WMS) provides a transparent, uniform interface for managing computing resources. The DIRAC Data Management System (DMS) offers all the necessary tools to ensure data handling operations: it supports transparent access to storage resources based on multiple technologies, and is easily expandable. Distributed Data management can be performed, also using third party services, and operations are resilient with respect to failures. DIRAC is highly customizable and can be easily extended. For these reasons, a vast and heterogeneous set of scientific collaborations have adopted DIRAC as the base for their computing models. Users from different experiments can interact with the system in different ways, depending on their specific tasks, expertise level and previous experience using command line tools, python APIs or Web Portals. The requirements of the diverse DIRAC user communities and hosting infrastructures triggered multiple developments to improve the system usability: examples include the adoption of industry standard authorization and authentication infrastructure solutions, the management of diverse computing resources (cloud, HPC, GPGPU, etc.), the handling of high-intensity work and data flows, but also advanced monitoring and accounting using no-SQL based solutions and message queues. This contribution will highlight DIRAC's current, upcoming and planned capabilities and technologies.

        Speaker: Federico Stagni (CERN)
      • 11:15
        Automated and Distributed Monte Carlo Generation for GlueX 15m

        MCwrapper is a set of system that manages the entire Monte Carlo production workflow for GlueX and provides standards for how that Monte Carlo is produced. MCwrapper was designed to be able to utilize a variety of batch systems in a way that is relatively transparent to the user, thus enabling users to quickly and easily produce valid simulated data at home institutions worldwide. Additionally, MCwrapper supports an autonomous system that takes user's project submissions via a custom web application. The system then atomizes the project into individual jobs, matches these jobs to resources, and monitors the jobs status. The entire system is managed by a database which tracks almost all facets of the systems from user submissions to the individual jobs themselves. Users can interact with their submitted projects online via a dashboard or, in the case of testing failure, can modify their project requests from a link contained in an automated email. Beginning in 2018 the GlueX Collaboration began to utilize the Open Science Grid (OSG) to handle a bulk of simulation tasks; these tasks are currently being performed on the OSG automatically via MCwrapper. This talk will outline the entire system of MCwrapper, its use cases, and the unique challenges facing the system.

        Speaker: Thomas Britton (JLab)
      • 11:30
        Production processing and workflow management software evaluation in the DUNE collaboration 15m

        The Deep Underground Neutrino Experiment (DUNE) will be the world’s foremost neutrino detector when it begins taking data in the mid-2020s. Two prototype detectors, collectively known as ProtoDUNE, have begun taking data at CERN and have accumulated over 3 PB of raw and reconstructed data since September 2018. Particle interaction within liquid argon time projection chambers are challenging to reconstruct, and the collaboration has set up a dedicated Production Processing group to perform centralized reconstruction of the large ProtoDUNE datasets as well as to generate large-scale Monte Carlo simulation. Part of the production infrastructure includes workflow management software and monitoring tools that are necessary to efficiently submit and monitor the large and diverse set of jobs needed to meet the experiment’s goals. We will give a brief overview of DUNE and ProtoDUNE, describe the various types of jobs within the Production Processing group’s purview, and discuss the software and workflow management strategies are currently in place to meet existing demand. We will conclude with a description of our requirements in a workflow management software solution and our planned evaluation process.

        Speaker: Dr Kenneth Richard Herner (Fermi National Accelerator Laboratory (US))
      • 11:45
        Evolution of the CMS Global Submission Infrastructure for the HL-LHC Era 15m

        Efforts in distributed computing of the CMS experiment at the LHC at CERN are now focusing on the functionality required to fulfill the projected needs for the HL-LHC era. Cloud and HPC resources are expected to be dominant relative to resources provided by traditional Grid sites, being also much more diverse and heterogeneous. Handling their special capabilities or limitations and maintaining global flexibility and efficiency, while also operating at scales much higher than the current capacity, are the major challenges being addressed by the CMS Submission Infrastructure team. This contribution will discuss the risks to the stability and scalability of the CMS HTCondor infrastructure extrapolated to such a scenario, thought to be derived mostly from its growing complexity, with multiple Negotiators and schedulers flocking work to multiple federated pools. New mechanisms for enhanced customization and control over resource allocation and usage, mandatory in this future scenario, will be also presented.

        Speaker: Antonio Perez-Calero Yzquierdo (Centro de Investigaciones Energéti cas Medioambientales y Tecno)
      • 12:00
        Efficient Iterative Calibration on the Grid using iLCDirac 15m

        Software tools for detector optimization studies for future experiments need to be efficient and reliable. One important ingredient of the detector design optimization concerns the calorimeter system. Every change of the calorimeter configuration requires a new set of overall calibration parameters which in its turn requires a new calorimeter calibration to be done. An efficient way to perform calorimeter calibration is therefor essential in any detector optimization tool set.
        In this contribution, we present the implementation of a calibration system in iLCDirac, which is an extension of the DIRAC grid interware. Our approach provides more direct control over the grid resources to reduce overhead of file download and job initialisation, and provides more flexibility during the calibration process. The service controls the whole chain of a calibration procedure, collects results from finished iterations and redistributes new input parameters among worker nodes. A dedicated agent monitors the health of running jobs and resubmits them if needed. Each calibration has an up-to-date backup which can be used for recovery in case of any disruption in the operation of the service.
        As a use case, we will present a study of optimization of the calorimetry system of the CLD detector concept for FCC-ee, which has been adopted from the CLICdet detector model. The detector has been simulated with the DD4hep package and calorimetry performance have been studied with the particle flow package PandoraPFA.

        Speaker: Andre Sailer (CERN)
      • 12:15
        New developments in cost modeling for the LHC computing 15m

        The increase in the scale of LHC computing during Run 3 and Run 4 (HL-LHC) will certainly require radical changes to the computing models and the data processing of the LHC experiments. The working group established by WLCG and the HEP Software Foundation to investigate all aspects of the cost of computing and how to optimise them has continued producing results and improving our understanding of this process. In particular, experiments have developed more sophisticated ways to calculate their resource needs, we have a much more detailed process to
        calculate infrastructure costs. This includes studies on the impact of HPC and GPU based resources on meeting the computing demands. We have also developed and perfected tools to quantitatively study the performance of experiments workloads and we are actively collaborating with other activities related to data access, benchmarking and technology cost evolution. In this contribution we expose our recent developments and results and outline the directions of future work.

        Speaker: Andrea Sciabà (CERN)
    • 11:00 12:30
      Track 4 – Data Organisation, Management and Access: community input, experiments and new perspectives Riverbank R8

      Riverbank R8

      Adelaide Convention Centre

      Convener: Xavier Espinal (CERN)
      • 11:00
        ServiceX – A Distributed, Caching, Columnar Data Delivery Service 15m

        We will describe a component of the Intelligent Data Delivery Service being developed in collaboration with IRIS-HEP and the LHC experiments. ServiceX is an experiment-agnostic service to enable on-demand data delivery specifically tailored for nearly-interactive vectorized analysis. This work is motivated by the data engineering challenges posed by HL-LHC data volumes and the increasing popularity of python and Spark-based analysis workflows.

        ServiceX gives analyzers the ability to query events by dataset metadata. It uses containerized transformations to extract just the data required for the analysis. This operation is collocated with the data lake to avoid transferring unnecessary branches over the WAN. Simple filtering operations are supported to further reduce the amount of data transferred.

        Transformed events are cached in a columnar datastore to accelerate delivery of subsequent similar requests. ServiceX will learn commonly related columns and automatically include them in the transformation to increase the potential for cache hits by other users.

        Selected events are streamed to the analysis system using an efficient wire protocol that can be readily consumed by a variety of computational frameworks. This reduces time-to-insight for physics analysis by delegating to ServiceX the complexity of event selection, slimming, reformatting, and streaming.

        Speaker: Benjamin Galewsky
      • 11:15
        OSiRIS: A Distributed Storage and Networking Project Update 15m

        We will report on the status of the OSiRIS project (NSF Award #1541335, UM, IU, MSU and WSU) after its fourth year. OSiRIS is delivering a distributed Ceph storage infrastructure coupled together with software-defined networking to support multiple science domains across Michigan’s three largest research universities. The project’s goal is to provide a single scalable, distributed storage infrastructure that allows researchers at each campus to work collaboratively with other researchers across campus or across institutions. The NSF CC*DNI DIBBs program which funded OSiRIS is seeking solutions to the challenges of multi-institutional collaborations involving large amounts of data and we are exploring the creative use of Ceph and networking to address those challenges.
        We will present details on the current status of the project and its various science domain users and use-cases. In the presentation we will cover the various design choices, configuration, tuning and operational challenges we have encountered in providing a multi-institutional Ceph deployment interconnected by a monitored, programmable network fabric. We will conclude with our plans for the final year of the project and its longer term outlook.

        Speaker: Shawn Mc Kee (University of Michigan (US))
      • 11:30
        Distributed data management on Belle II 15m

        The Belle II experiment started taking physics data in March 2019, with an estimated dataset of order 60 petabytes expected by the end of operations in the mid-2020s. Originally designed as a fully integrated component of the BelleDIRAC production system, the Belle II distributed data management (DDM) software needs to manage data across 70 storage elements worldwide for a collaboration of nearly 1000 physicists. By late 2018, this software required significant performance improvements to meet the requirements of physics data taking and was seriously lacking in automation. Rucio, the DDM solution created by ATLAS, was an obvious alternative but required tight integration with BelleDIRAC and a seamless yet non-trivial migration. This contribution describes the work done on both DDM options, the current status of the software running successfully in production and the problems associated with trying to balance long-term operations cost against short term risk.

        Speaker: Siarhei Padolski (BNL)
      • 11:45
        Jiskefet, a bookkeeping application for ALICE 15m

        A new bookkeeping system called Jiskefet is being developed for A Large Ion Collider Experiment (ALICE) during Long Shutdown 2, to be in production until the end of LHC Run 4 (2029).

        Jiskefet unifies two functionalities. The first is gathering, storing and presenting metadata associated with the operations of the ALICE experiment. The second is tracking the asynchronous processing of the physics data.

        It will replace the existing ALICE Electronic Logbook and AliMonitor, allowing for a technology refresh and the inclusion of new features based on the experience collected during Run 1 and Run 2.

        The front end leverages web technologies much in use nowadays such as TypeScript and NodeJS and is adaptive to various clients such as tablets, mobile device and other screens. The back end includes a Swagger based REST API and a relational database.

        This paper will describe the current status of the development, the initial experience in detector standalone commissioning setups and the future plans. It will also describe the organization of the work done by various student teams who work on Jiskefet in sequential and parallel semesters and how continuity is guaranteed by using guidelines on coding, documentation and development.

        Speaker: Marten Teitsma (Amsterdam University of Applied Sciences (NL))
      • 12:00
        Development of the JUNO Conditions Data Management System 15m

        (On behalf of the JUNO collaboration)

        Abstract:
        The JUNO (Jiangmen Underground Neutrino Observatory) experiment is designed to determine the neutrino mass hierarchy and precisely measure oscillation parameters with an unprecedented energy resolution of 3% at 1MeV. It is composed of a 20kton liquid scintillator central detector equipped with 18000 20” PMTs and 25000 3” PMTs, a water pool with 2000 20” PMTs, and a top tracker. Conditions data, coming from calibration and detector monitoring, are heterogeneous, different type of conditions data has different write rates, data format and data volume. JUNO conditions data management system (JCDMS) is developed to homogeneously treat all these heterogeneous conditions data in order to provide easy management and access with both Restful API and web interfaces, support good scalability and maintenance for long time running. we will present the status and development of JCDMS including the data model, workflows, interfaces, data caching and performance of the system.

        Speaker: Prof. Xingtao Huang (Shandong University)
      • 12:15
        Evaluation of the ATLAS model for remote access to database resident information for LHC Run 3 15m

        The ATLAS model for remote access to database resident information relies upon a limited set of dedicated and distributed Oracle database repositories complemented with the deployment of Frontier system infrastructure on the WLCG. ATLAS clients with network access can get the database information they need dynamically by submitting requests to a squid server in the Frontier network which provides results from its cache or passes new requests along the network to launchpads co-located at one of the Oracle sites (the master Oracle database at CERN or one of the Tier 1 Oracle database replicas). Since the beginning of LHC Run 1, the system has evolved in terms of client, squid, and launchpad optimizations but the distribution model has remained fundamentally unchanged.
        On the whole, the system has been broadly successful in providing data to clients with relatively few disruptions even while site databases were down due to redundancy overall. At the same time, its quantitative performance characteristics, such as the global throughput of the system, the load distribution between sites, and the constituent interactions that make up the whole, were largely unknown. But more recently, information has been collected from launchpad and squid logs into an Elastic Search repository which has enabled a wide variety of studies of various aspects of the system.
        This presentation will describe dedicated studies of the data collected in Elastic Search over the previous year to evaluate the efficacy of the distribution model. Specifically, we will quantify any advantages that the redundancy of the system offers as well as related aspects such as the geographical dependence of wait times seen by clients in getting a response to its requests. These studies are essential so that during LS2 (the long shutdown between LHC Run 2 and Run 3), we can adapt the system in preparation for the expected increase in the system load in the ramp up to Run 3 operations.

        Speaker: Elizabeth Gallas (University of Oxford (GB))
    • 11:00 12:30
      Track 5 – Software Development: General frameworks and schedulers Riverbank R2

      Riverbank R2

      Adelaide Convention Centre

      Convener: Dr Martin Ritter (LMU / Cluster Universe)
      • 11:00
        A software framework for FCC studies: status and plans 15m

        The Future Circular Collider (FCC) is designed to provide unprecedented luminosity and unprecedented centre-of-mass energies. The physics reach and potential of the different FCC options - $e^+e^-$, $pp$, $e^-p$ - has been studied and published in dedicated Conceptual Design Reports (CDRs) published at the end of 2018.
        Conceptual detector designs have been developed for such studies and tested with a mixture of fast and full simulations. These investigations have conducted using a common software framework called FCCSW.
        In this presentation, after summarising the improvements implemented in FCCSW to achieve the results included in the CDRs, we will present the current development plans to support the continuation of the physics potential and detector concept optimization studies in view of future strategic decisions, in particular for the electron-positron machine.

        Speaker: Gerardo Ganis (CERN)
      • 11:15
        LArSoft and Future Framework Directions at Fermilab 15m

        The diversity of the scientific goals across HEP experiments necessitates unique bodies of software tailored for achieving particular physics results. The challenge, however, is to identify the software that must be unique, and the code that is unnecessarily duplicated, which results in wasted effort and inhibits code maintainability.

        Fermilab has a history of supporting and developing software projects that are shared among HEP experiments. Fermilab's scientific computing division currently expends effort in maintaining and developing the LArSoft toolkit, used by liquid argon TPC experiments, as well as the event-processing framework technologies used by LArSoft, CMS, DUNE, and the majority of Fermilab-hosted experiments. As computing needs for DUNE and the HL-LHC become clearer, the computing models are being rethought. This talk will focus on Fermilab's plans for addressing the evolving software landscape as it relates to LArSoft and the event-processing frameworks, and how commonality among experiment software can be achieved while still supporting customizations necessary for a given experiment's physics goals.

        Speaker: Christopher Jones (Fermi National Accelerator Lab. (US))
      • 11:30
        ALFA: A framework for building distributed applications 15m

        The ALFA framework is a joint development between ALICE Online-Offline and FairRoot teams. ALFA has a distributed architecture, i.e. a collection of highly maintainable, testable, loosely coupled, independently deployable processes.

        ALFA allows the developer to focus on building single-function modules with well-defined interfaces and operations. The communication between the independent processes is handled by FairMQ transport layer. FairMQ offers multiple implementations of its abstract data transport interface, it integrates some popular data transport technologies like ZeroMQ and nanomsg. But also provides shared memory and RDMA transport (based on libfabric) for high throughput, low latency applications. Moreover, FairMQ allows the single process to use multiple and different transports at the same time.

        FairMQ based processes can be controlled and orchestrated via different systems by implementing the corresponding plugin. However, ALFA delivers also the Dynamic Deployment System (DDS) as an independent set of utilities and interfaces, providing a dynamic distribution of different user processes on any Resource Management System (RMS) or a laptop.

        ALFA is already being tested and used by different experiments in different stages of data processing as it offers an easy integration of heterogeneous hardware and software. Examples of ALFA usage in different stages of event processing will be presented; in a detector read-out as well as in an online reconstruction and in a pure offline world of detector simulations.

        Speaker: Mohammad Al-Turany (GSI - Helmholtzzentrum fur Schwerionenforschung GmbH (DE))
      • 11:45
        Using OpenMP for HEP Framework Algorithm Scheduling 15m

        The OpenMP standard is the primary mechanism used at high performance computing facilities to allow intra-process parallelization. In contrast, many HEP specific software (such as CMSSW, GaudiHive, and ROOT) make use of Intel's Threading Building Blocks (TBB) library to accomplish the same goal. In this talk we will discuss our work to compare TBB and OpenMP when used for scheduling algorithms to be run by a HEP style data processing framework (i.e. running hundreds of interdependent algorithms at most once for each event read from the detector). This includes both scheduling of different algorithms to be run concurrently as well as scheduling concurrent work within one algorithm. As part of the discussion we present an overview of the OpenMP threading model. We also explain how we used OpenMP when creating a simplified HEP-like processing framework. Using that simplified framework, and a similar one written using TBB, we will present performance comparisons between TBB and different compiler versions of OpenMP.

        Speaker: Christopher Jones (Fermi National Accelerator Lab. (US))
      • 12:00
        Configuration and scheduling of the LHCb trigger application 15m

        The high-level trigger (HLT) of LHCb in Run 3 will have to process 5 TB/s of data, which is about two orders of magnitude larger compared to Run 2. The second stage of the HLT runs asynchronously to the LHC, aiming for a throughput of about 1 MHz. It selects analysis-ready physics signals by O(1000) dedicated selections totaling O(10000) algorithms to achieve maximum efficiency. This poses two problems: correct configuration of the application and low-overhead execution of individual algorithms and evaluation of the decision logic.

        A python-based system for configuring the data and control flow of the Gaudi-based application, including all components, is presented. It is designed to be user-friendly by using functions for modularity and removing indirection layers employed previously in Run 2. Robustness is achieved by fully eliminating global state and instead building the data flow graph in a functional manner while keeping configurability of the full call stack.

        A prototype of the second HLT stage comprising all recent features including a new scheduling algorithm, a faster data store and the above mentioned configuration system is benchmarked, demonstrating the performance of the framework with the expected application complexity.

        Speaker: Niklas Nolte (CERN / Technische Universitaet Dortmund (DE))
      • 12:15
        Allen: A software framework for the GPU High Level Trigger 1 of LHCb 15m

        As part of the LHCb detector upgrade in 2021, the hardware-level trigger will be removed, coinciding with an increase in luminosity. As a consequence, about 40 Tbit/s of data will be processed in a full-software trigger, a challenge that has prompted the exploration of alternative hardware technologies. Allen is a framework that permits concurrent many-event execution targeting many-core architectures. We present the core infrastructure of this R&D project developed in the context of the LHCb Upgrade I. Data transmission overhead is hidden with a custom memory manager, and GPU resource usage is maximized employing a deterministic scheduler. Our framework is extensible and covers the control flow and data dependency requirements of the LHCb High Level Trigger 1 algorithms. We discuss the framework design, performance and integration aspects of a full realization of a GPU High Level Trigger 1 in LHCb.

        Speaker: Daniel Hugo Campora Perez (Universidad de Sevilla (ES))
    • 11:00 12:30
      Track 6 – Physics Analysis: Analysis Tools & Methods Hall G

      Hall G

      Adelaide Convention Centre

      Convener: Ross Young (University of Adelaide)
      • 11:00
        zfit: scalable pythonic fitting 15m

        Statistical modelling is a key element for High-Energy Physics (HEP) analysis. Currently, most of this modelling is performed with the ROOT/RooFit toolkit which is written in C++ and provides Python bindings which are only loosely integrated into the scientific Python ecosystem. We present zfit, a new alternative to RooFit, written in pure Python. Built on top of TensorFlow (a modern, high level computing library for massive computations), zfit provides a high level interface for advanced model building and fitting. It is also designed to be extendable in a very simple way, allowing the usage of cutting-edge developments from the scientific Python ecosystem in a transparent way. In this talk, the main features of zfit are introduced, and its extension to data analysis, especially in the context of HEP experiments, is discussed.

        Speaker: Jonas Eschle (Universitaet Zuerich (CH))
      • 11:15
        Machine Learning with ROOT/TMVA 15m

        ROOT provides, through TMVA, machine learning tools for data analysis at HEP experiments and beyond. In this talk, we present recently included features in TMVA and the strategy for future developments in the diversified machine learning landscape. Focus is put on fast machine learning inference, which enables analysts to deploy their machine learning models rapidly on large scale datasets. The new developments are paired with newly designed C++ and Python interfaces supporting modern C++ paradigms and full interoperability in the Python ecosystem.

        We present as well a new deep learning implementation for convolutional neural network using the cuDNN library for GPU. We show benchmarking results in term of training time and inference time, when comparing with other machine learning libraries such as Keras/Tensorflow.

        Speaker: Stefan Wunsch (KIT - Karlsruhe Institute of Technology (DE))
      • 11:30
        Evolution of web-based analysis for Machine Learning and LHC Experiments: power of integrating storage, interactivity and collaboration with JupyterLab in SWAN 15m

        SWAN (Service for Web-based ANalysis) is a CERN service that allows users to perform interactive data analysis in the cloud, in a "software as a service" model. The service is a result of the collaboration between IT Storage and Databases groups and EP-SFT group at CERN. SWAN is built upon the widely-used Jupyter notebooks, allowing users to write - and run - their data analysis using only a web browser. SWAN is a data analysis hub: users have immediate access to user storage CERNBox, entire LHC data repository on EOS, software (CVMFS) and computing resources, in a pre-configured, ready-to-use environment. Sharing of notebooks is fully integrated with CERNBox and users can easily access their notebook projects on all devices supported by CERNBox.

        In the first quarter of 2019 we have recorded more than 1300 individual users of SWAN, with a majority from all four LHC experiments. Integration of SWAN with CERN Spark clusters is at the core of the new controls data logging system for the LHC. Every month new users discover SWAN through tutorials on data analysis and machine learning.

        The SWAN service evolves, driven by the user's needs. In the future SWAN will provide access to GPUs, to the more powerful interface of Jupyterlab - that replaces Jupyter notebooks - and to a more configurable, easier to use and more shareable way of setting the software environment of Projects and notebooks.

        This presentation will update the HEP community with the status of this effort and its future direction, together with the general evolution of SWAN.

        Speaker: Jakub Moscicki (CERN)
      • 11:45
        The F.A.S.T. toolset: Using YAML to make tables out of trees 15m

        The Faster Analysis Software Taskforce (FAST) is a small, European group of HEP researchers that have been investigating and developing modern software approaches to improve HEP analyses. We present here an overview of the key product of this effort: a set of packages that allows a complete implementation of an analysis using almost exclusively YAML files. Serving as an analysis description language (ADL), this toolset builds on top of the evolving technologies from the Scikit-HEP and IRIS-HEP projects as well as industry-standard libraries such as Pandas and Matplotlib. Data processing starts with event-level data (the trees) and can proceed by adding variables, selecting events, performing complex user-defined operations and binning data, as defined in the YAML description. The resulting outputs (the tables) are stored as Pandas dataframes or FlatBuffers defined by Aghast, which can be programmatically manipulated. The F.A.S.T. tools can then convert these into plots or inputs for fitting frameworks. No longer just a proof-of-principle, these tools are now being used in CMS analyses, the LUX-ZEPLIN experiment, and by students on several other experiments. In this talk we will showcase these tools through examples, highlighting how they address the different experiments’ needs, and compare them to other similar approaches.

        Speaker: Benjamin Krikler (University of Bristol (GB))
      • 12:00
        High-dimensional data visualisation with the grand tour 15m

        In physics we often encounter high-dimensional data, in the form of multivariate measurements or of models with multiple free parameters. The information encoded is increasingly explored using machine learning, but is not typically explored visually. The barrier tends to be visualising beyond 3D, but systematic approaches for this exist in the statistics literature. I will use examples from particle and astrophysics to show how we can use the “grand tour” for such multidimensional visualisations, for example to explore grouping in high dimension and for visual identification of multivariate outliers. I will then discuss the idea of projection pursuit, i.e. searching the high-dimensional space for “interesting” low dimensional projections, and illustrate how we can detect complex associations between multiple parameters.

        Speaker: Ursula Laa (Monash University)
      • 12:15
        Faster RooFitting: Automated Parallel Computation of Collaborative Statistical Models 15m

        RooFit is the statistical modeling and fitting package used in many experiments to extract physical parameters from reduced particle collision data. RooFit aims to separate particle physics model building and fitting (the users' goals) from their technical implementation and optimization in the back-end. In this talk, we outline our efforts to further optimize the back-end by automatically running major parts of user models in parallel on multi-core machines.

        A major challenge is that RooFit allows users to define many different types of models, with different types of computational bottlenecks. Our automatic parallelization framework must then be flexible, while still reducing run-time by at least an order of magnitude, preferably more.

        We have performed extensive benchmarks and identified at least three bottlenecks that will benefit from parallelization. To tackle these and possible future bottlenecks, we designed a parallelization layer that allows us to parallelize existing classes with minimal effort, but with high performance and retaining as much of the existing class's interface as possible.

        The high-level parallelization model is a task-stealing approach. Our multi-process approach uses ZeroMQ socket-based communication. Preliminary results show speed-ups of factor 2 to 20, depending on the exact model and parallelization strategy.

        We will integrate our parallelization layer into RooFit in such a way that impact on the end-user interface is minimal. This constraint, together with new features introduced in a concurrent RooFit project on vectorization and dataflow redesign, warrants a redesign of the RooFit internal classes for likelihood evaluation and other test statistics. We will briefly outline the implications of this for users.

        Speaker: Dr Carsten Daniel Burgard (Nikhef National institute for subatomic physics (NL))
    • 11:00 12:30
      Track 7 – Facilities, Clouds and Containers: Monitoring and benchmarking Riverbank R7

      Riverbank R7

      Adelaide Convention Centre

      Convener: Oksana Shadura (University of Nebraska Lincoln (US))
      • 11:00
        Machine Learning-based Anomaly Detection of Ganglia Monitoring data in HEP Data Center 15m

        The IHEP local cluster is a middle-sized HEP data center which consists of 20’000 CPU slots, hundreds of data servers, 20 PB disk storage and 10 PB tape storage. After data taking of JUNO and LHAASO experiment, the data volume processed at this center will approach 10 PB data per year. Facing the current cluster scale, anomaly detection is a non-trivial task in daily maintenance. Traditional methods such as static thresholding of performance metrics, key words searching in system logs, etc., require expertise of certain software systems, and cannot be easy to transplant. Besides, these methods cannot easily adapt to the changes of workloads and hardware configurations. Anomalies are data points which are either different from the majority of others or different from the expectation of a reliable prediction model in a time series. With a sufficient training sample dataset, machine learning-based anomaly detections which leverage these statistical characteristics can largely avoid the disadvantages of traditional methods. The Ganglia monitoring system at IHEP collects billions of timestamped monitoring data from the cluster every year. It provides sufficient data samples to train machine learning models. In this presentation, we firstly developed a generic anomaly detection framework to facilitate different detection task. It facilities common tasks such as data sample building, retagging and visualization, model calling, deviation measurement and performance measurement in machine learning-based anomaly detection methods. Then, for massive storage system, we developed and trained a spatial anomaly detection model based on Isolation Forest algorithm and a time series anomaly detection model based on LSTM recurrent neural networks to validate our idea. Initial performance comparison of our methods and traditional methods will be provided at the end of the presentation.

        Speakers: Ms Juan Chen (IHEP), juan chen (IHEP)
      • 11:15
        Anomaly detection using Unsupervised Machine Learning for Grid computing site operation 15m

        A Grid computing site consists of various services including Grid middlewares, such as Computing Element, Storage Element and so on. Ensuring a safe and stable operation of the services is a key role of site administrators. Logs produced by the services provide useful information for understanding the status of the site. However, it is a time-consuming task for site administrators to monitor and analyze the service logs everyday. Therefore, a support framework (gridalert), which detects anomaly logs and alerts to site administrators, has been developed using Machine Learning techniques.

        Typical classifications using Machine Learning require pre-defined labels. It is difficult to collect a large amount of anomaly logs to build a Machine Learning model that covers all possible pre-defined anomalies. Therefore, Unsupervised Machine Learning based on clustering algorithms is used in the gridalert to detect anomaly logs. Several clustering algorithms, such as k-means, DBSCAN and IsolationForest, and its parameters have been compared in order to maximize the performance of the anomaly detection for Grid computing site operations. The gridalert has been deployed to Tokyo Tier2 site, which is one of the Worldwide LHC Computing Gird sites, and is used in operation. In this presentation, studies about Machine Learning algorithms for the anomaly detection and our operational experiences of the gridalert will be reported.

        Speaker: Tomoe Kishimoto (University of Tokyo (JP))
      • 11:30
        Using HEP experiment workflows for the benchmarking and accounting of computing resources 15m

        The benchmarking and accounting of CPU resources in WLCG has been based on the HEP-SPEC06 (HS06) suite for over a decade. HS06 is stable, accurate and reproducible, but it is an old benchmark and it is becoming clear that its performance and that of typical HEP applications have started to diverge. After evaluating several alternatives for the replacement of HS06, the HEPIX benchmarking WG has chosen to focus on the development of a HEP-specific suite based on actual software workloads of the LHC experiments, rather than on a standard industrial benchmark like the new SPEC CPU 2017 suite.

        This presentation will describe the motivation and implementation of this new benchmark suite, which is based on container technologies to ensure portability and reproducibility. This approach is designed to provide a better correlation between the new benchmark and the actual production workloads of the experiments. It also offers the possibility to separately explore and describe the independent architectural features of different computing resource types, which is expected to be increasingly important with the growing heterogeneity of the HEP computing landscape. In particular, an overview of the initial developments to address the benchmarking of non-traditional computing resources such as HPCs and GPUs will also be provided.

        Speaker: Andrea Valassi (CERN)
      • 11:45
        WLCG Networks: Update on Monitoring and Analytics 15m

        WLCG relies on the network as a critical part of its infrastructure and therefore needs to guarantee effective network usage and prompt detection and resolution of any network issues, including connection failures, congestion and traffic routing. The OSG Networking Area, in partnership with WLCG, is focused on being the primary source of networking information for its partners and constituents. It was established to ensure sites and experiments can better understand and fix networking issues, while providing an analytics platform that aggregates network monitoring data with higher level workload and data transfer services. This has been facilitated by the global network of the perfSONAR instances that have been commissioned and are operated in collaboration with WLCG Network Throughput Working Group. An additional important update is the inclusion of the newly funded NSF project SAND (Service Analytics and Network Diagnosis) which is focusing on network analytics.

        In this talk we'll describe the current state of the network measurement and analytics platform and summarise the activities taken by the working group and our collaborators, focusing mainly on the throughput issues that have been reported and resolved during the recent period with the help of the perfSONAR network. We will also cover the updates on the higher level services that were developed to help bring perfSONAR network to its full potential. This includes the progress being made in providing higher level analytics, alerting and alarming from the rich set of network metrics we are gathering. . Finally, we will discuss and propose potential R&D areas related to improving the network throughput in general as well as prepare the infrastructure for the foreseen major changes in the way network will be provisioned and operated in the future.

        Speaker: Pedro Andrade (CERN)
      • 12:00
        WLCG Dashboards with Unified Monitoring 15m

        Monitoring of the CERN Data Centres and the WLCG infrastructure is now largely based on the MONIT infrastructure provided by CERN IT. This is the result of the migration from several old in-house developed monitoring tools into a common monitoring infrastructure based on open source technologies such as Collectd, Flume, Kafka, Spark, InfluxDB, Grafana and others. The MONIT infrastructure relies on CERN IT services (OpenStack, Puppet, Gitlab, DBOD, etc) and covers the full range of monitoring tasks: metrics and logs collection, alarms generation, data validation and transport, data enrichment and aggregation (where applicable), dashboards visualisation, reports generation, etc. This contribution will present the different services offered by the MONIT infrastructure today, highlight the main monitoring use cases from the CERN Data Centres, WLCG, and Experiments, and analyse the last years experience of moving from legacy well-established custom monitoring tools into a common open source-based infrastructure.

        Speaker: Pedro Andrade (CERN)
      • 12:15
        Large Elasticsearch cluster management 15m

        The Centralised Elasticsearch Service at CERN runs the infrastructure to
        provide Elasticsearch clusters for more than 100 different use cases.

        This contribution presents how the infrastructure is managed, covering the
        resource distribution, instance creation, cluster monitoring and user
        support. The contribution will present the components that have been identified as
        critical in order to share resources and minimize the amount of clusters and
        machines needed to run the service.In particular, all the automation for the
        instance configuration, including index template management, backups and
        Kibana settings, will be explained in detail.

        Speaker: Pablo Saiz (CERN)
    • 11:00 12:30
      Track 8 – Collaboration, Education, Training and Outreach: Collaborative tools Riverbank R1

      Riverbank R1

      Adelaide Convention Centre

      Convener: Tibor Simko (CERN)
      • 11:00
        Challenges and opportunities when migrating CERN e-mail system to open source 15m

        E-mail service is considered as a critical collaboration system. We will share our experience regarding technical and organizational challenges when migrating 40 000 mailboxes from Microsoft Exchange to free and open source software solution: Kopano.

        Speaker: Pawel Grzywaczewski (CERN)
      • 11:15
        Experience finding MS Project Alternatives at CERN 15m

        As of March 2019, CERN is no longer eligible for academic licences of Microsoft products. For this reason, CERN IT started a series of task forces to respond to the evolving requirements of the user community with the goal of reducing as much as possible the need for Microsoft licensed software. This exercise was an opportunity to understand better the user requirements for all office applications. Here we focus on MS Project, the dominant PC-based project management software, which has been used at CERN for many years. There were over 1500 installations at CERN when the task force started, with a heterogeneous pool of users in terms of required functionality, area of work and expertise. This paper will present an evaluation of users’ needs and whether they could be fulfilled with cheaper and less advanced solutions for project management and scheduling. Moreover, selected alternatives, their deployment and lessons learned will be described in more detail. Finally, it will present the approach on how to communicate, train and migrate users to the proposed solutions.

        Speakers: Maria Alandes Pradillo (CERN), Sebastian Bukowiec (CERN)
      • 11:30
        eXtreme monitoring: CERN video conference system and audio-visual IoT device infrastructure 15m

        In this talk the approach chosen to monitor firstly a world-wide video conference server infrastructure and secondly a wide diversity of audio-visual devices that build up the audio-visual conference room ecosystem at CERN will be presented.

        CERN video conference system is a complex ecosystem which is being used by most HEP institutes, together with Swiss Universities through SWITCH. As a proprietary platform, on its on-premise version, Vidyo offers a very limited monitoring. In order to improve support to our user community together with a better understanding of the Vidyo platform for service managers and video conference supporters a set of tools to monitor the system has been developed keeping in mind simplicity, flexibility, maintainability and cost efficiency reusing as much as possible technologies offered by IT services: Elasticsearch stack, Influxdb, Openshift, Kubernetes, Openstack, etc. The result is a set of dashboards that greatly simplify access to information required by CERN IT helpdesk and service managers and that could be provided to the users. Most of the components developed are open source [1,2], and could be reused for services facing similar problems.

        With the arrival of IP devices in the Audio-Visual and Conferencing (AVC) equipment, the possibilities to develop an agnostic solution for monitoring this IoT jungle (video encoders, videoconference codecs, screens, projectors, microphones, clocks,..) becomes feasible. After trying with no real success existing commercial products for monitoring, CERN is now developing an opensource solution to effectively monitor/operate the AVC ecosystem using existing opensource components and central services provided by the IT deparment: node-red/mqtt, telegraf/influxdb/grafana, beats/logstash/elasticseach/kibana, openshift, etc.

        [1] https://github.com/CERNCDAIC/aggsvidyo
        [2] https://github.com/CERNCDAIC/resthttpck

        Speaker: Ruben Domingo Gaspar Aparicio (CERN)
      • 11:45
        Development of a Versatile, Full-Featured Search Functionality for Indico 15m

        Indico, CERN’s popular open-source tool for event management, is in widespread use among facilities that make up the HEP community. It is extensible through a robust plugin architecture that provides features such as search and video conferencing integration. In 2018, Indico version 2 was released with many notable improvements, but without a full-featured search functionality that could be implemented easily outside of CERN. At both Fermi and Brookhaven National Laboratories, the user community viewed the lack of this popular feature as a significant impediment to deployment of the new software. In the meantime, CERN embarked upon a major redesign of their core search service, one that would also necessitate a rewrite of the Indico search interface. Seeing this pressing need, the two US labs decided to collaborate, with assistance from the CERN development team, on a project to develop the requisite search functionality for the larger user community. The resulting design exploits the simplified schema defined in the new CERN Search micro-service, based on Invenio and Elasticsearch, while still providing a flexible path to implementation for alternative backend search services. It is intended to provide a software package that can be installed easily and used out of the box, by anyone at any site. This presentation will discuss the design choices and architectural challenges, and provide an overview of the deployment and use of these new plugins.

        Speaker: Penelope Constanta (Fermilab)
      • 12:00
        Evolution of the CERNBox platform to support collaborative applications and MALT 15m

        CERNBox is the CERN cloud storage hub for more than 16000 users at CERN. It allows synchronising and sharing files on all major desktop and mobile platforms (Linux, Windows, MacOSX, Android, iOS) providing universal, ubiquitous, online- and offline access to any data stored in the CERN EOS infrastructure. CERNBox also provides integration with other CERN services for big science: visualisation tools, interactive data analysis and real-time collaborative editing.

        Over the last two years, CERNBox has evolved from a pure cloud sync and share platform into a collaborative service, to support new applications such as DrawIO for diagrams and organigrams sketching, OnlyOffice and Collabora Online for documents editing, and DXHTML Gantt for project management, as alternatives to traditional desktop applications. Moving to open source applications has the advantage to reduce licensing costs and enables easier integration within the CERN infrastructure.

        Leveraging large and diverse set of applications at CERN, we propose a bring-your-own-application model where user groups can easily integrate their specific web-based applications in all available CERNBox workflows. We report on our experience managing such integrations and applicable use-cases, also in a broader scientific context where an emerging community including other institutes and SMEs is evolving the standards for sync & share storage.

        Speaker: Hugo Gonzalez Labrador (CERN)
      • 12:15
        Operating the Belle II Collaborative Services and Tools 15m

        Collaborative services are essential for any experiment.
        They help to integrate global virtual communities by allowing to share
        and exchange relevant information among members.
        Typical examples are public and internal web pages, wikis, mailing
        list services, issue tracking systems, and services for meeting
        organizations and documents.

        After reviewing their collaborative services with respect to security,
        reliability, availability, and scalability, the Belle II collaboration
        decided in 2016 to migrate services and tools into the existing IT
        infrastructure at DESY. So far missing services were added and
        workflows adapted. As a new development, a membership management
        system which serves the needs of a global collaboration as of to-date
        with 968 scientists of more than 113 institutions in 25 countries all
        around the world, was put into operation in the beginning of 2018.

        Almost all essential service of a living collaboration were subject to
        modifications, some of them with major or complete changes. Moreover,
        an already productive collaboration had to give up accustomed systems
        and adopt to new ones with new look-and-feels and more restrictive
        security rules.

        In the contribution to CHEP2019 we will briefly review the planning
        and realization of the migration process and thoroughly discuss
        experiences which we gained while supporting the daily work of the
        Belle II collaboration.

        Speaker: Thomas Kuhr (Ludwig Maximilians Universitat (DE))
    • 11:00 12:30
      Track 9 – Exascale Science: HPC facilities Riverbank R4

      Riverbank R4

      Adelaide Convention Centre

      Convener: Fabio Hernandez (IN2P3 / CNRS computing centre)
      • 11:00
        Extension of the INFN Tier-1 on a HPC system 15m

        The INFN Tier-1 located at CNAF in Bologna (Italy) is a major center of the WLCG e-Infrastructure, supporting the 4 major LHC collaborations and more than 30 other INFN-related experiments.
        After multiple tests towards elastic expansion of CNAF compute power via Cloud resources (provided by Azure, Aruba and in the framework of the HNSciCloud project), but also building on the experience gained with the production quality extension of the Tier-1 farm on remote owned sites, the CNAF team, in collaboration with experts from the ATLAS, CMS, and LHCb experiments, has been working to put in production a solution of an integrated HTC+HPC system with the PRACE CINECA center, located nearby Bologna. Such extension will be implemented on the Marconi A2 partition, equipped with Intel Knights Landing (KNL) processors. A number of technical challenges were faced and solved in order to successfully run on low RAM nodes, as well as to overcome the closed environment (network, access, software distribution, … ) that HPC systems deploy with respect to standard GRID sites. We show preliminary results from a large scale integration effort, using resources secured via the successful PRACE grant N. 2018194658, for 30 million KNL core hours.

        Speaker: Tommaso Boccali (Universita & INFN Pisa (IT))
      • 11:15
        Large-scale HPC deployment of Scalable CyberInfrastructure for Artificial Intelligence and Likelihood Free Inference (SCAILFIN) 15m

        The NSF-funded Scalable CyberInfrastructure for Artificial Intelligence and Likelihood Free Inference (SCAILFIN) project aims to develop and deploy artificial intelligence (AI) and likelihood-free inference (LFI) techniques and software using scalable cyberinfrastructure (CI) built on top of existing CI elements. Specifically, the project has extended the CERN-based REANA framework, a cloud-based data analysis platform deployed on top of Kubernetes clusters that was originally designed to enable analysis reusability and reproducibility. REANA is capable of orchestrating extremely complicated multi-step workflows, and uses Kubernetes clusters both for scheduling and distributing container-based workloads across a cluster of available machines, as well as instantiating and monitoring the concrete workloads themselves.

        This work describes the challenges and development efforts involved in extending REANA and the components that were developed in order to enable large scale deployment on High Performance Computing (HPC) resources.

        Using the Virtual Clusters for Community Computation (VC3) infrastructure as a starting point, we implemented REANA to work with a number of differing workload managers, including both high performance and high throughput, while simultaneously removing REANA's dependence on Kubernetes support at the workers level. Performance results derived from running AI/LFI training workflows on a variety of large HPC sites will be presented.

        Speaker: Mike Hildreth (University of Notre Dame (US))
      • 11:30
        EuroEXA: an innovative and scalable FPGA-based system for extreme scale computing in Europe 15m

        Nowadays, a number of technology R&D activities has been launched in Europe trying to close the gap with traditional HPC providers like USA and Japan and more recently emerging ones like China.
        The EU HPC strategy, funded through EuroHPC initiative, leverages on two different pillars: the first one targets the procurement and the hosting of two/three commercial pre-Exascale systems, in order to provide the HPC user community with world-level class computing systems; the second one aims at boosting industry-research collaboration in order to design a new generation of Exascale systems which is to be mainly based on European technology.
        In this framework, analysis and validation of the HPC-enabling technologies is a very critical task and the FETHPC H2020 EuroEXA project (https://euroexa.eu) is prototyping a medium size (but scalable to extreme level) computing platform as proof-of-concept of an EU-designed HPC system.
        EuroEXA exploits FPGA devices, with their ensemble of either standard and custom high-performance interfaces, DSP blocks for task acceleration and a huge number of user- assigned logic cells. FPGA adoption allows us to design European innovative IPs such as application-tailored acceleration hardware (for high performances in the computing node) and low latency, high throughput custom network (for scalability).
        The EuroEXA computing node is based on a single module hosting Xilinx UltraScale+ FPGAs for application code acceleration hardware, control and network implementation, and, in a later phase, even a new project-designed, ARM-based, low power multi-core chip.
        EuroEXA interconnect is an FPGA-based hierarchical hybrid network characterized by direct topology at “blade” level (16 computing nodes on a board) and a custom switch, implementing a mix of full-crossbar and Torus topology, for interconnection with the upper levels.
        EuroEXA will also introduce a new high density liquid-cooling technology for blade system and a new multirack modular assembly based on standard shipping containers in order to provide an effective solution for moving, placing and operating large scale EuroEXA system.
        A complete and system-optimized programming software stack is under design and a number of scientific, engineering and AI-oriented applications are used to co-design, benchmark and validate the EuroEXA hardware/software solutions.
        In this talk, we will introduce the main project motivations and goals, its positioning within the EuroHPC landscape, the status of hardware and software development and the possible synergies with HEP computing requirements.

        Speaker: Piero Vicini (Sapienza Universita e INFN, Roma I (IT))
      • 11:45
        Using HEP workloads to optimize requirements for procurement of a future HPC facility 15m

        The Dutch science funding organization NWO is in the process of drafting requirements for the procurement of a future high-performance compute facility. To investigate the requirements for this facility to potentially support high-throughput workloads in addition to traditional high-performance workloads, a broad range of HEP workloads are being functionally tested on the current facility. The requirements obtained from this pilot will be presented, together with technical issues solved and requirements on HPC and HEP software and support.

        Speaker: Roel Aaij (Nikhef National institute for subatomic physics (NL))
      • 12:00
        Deep Learning for HEP on HPC at NERSC 15m

        We present recent work in supporting deep learning for particle physics and cosmology at NERSC, the US Dept. of Energy mission HPC center. We describe infrastructure and software to support both large-scale distributed training across (CPU and GPU) HPC resources and for productive interfaces via Jupyter notebooks. We also detail plans for accelerated hardware for deep learning in the future HPC machines at NERSC, ‘Perlmutter’ and beyond. We demonstrate these capabilities with a characterisation of the emerging deep learning workload running at NERSC. We also present use of these resources to implement specific cutting-edge applications including conditional Generative Adversarial Networks for particle physics and dark-matter cosmology simulations and bayesian inference via probabilistic programming for LHC analyses.

        Speaker: Steven Farrell (Lawrence Berkeley National Lab (US))
    • 12:30 14:00
      Lunch 1h 30m
    • 14:00 15:30
      Track 1 – Online and Real-time Computing: Monitoring and control systems Riverbank R5

      Riverbank R5

      Adelaide Convention Centre

      Convener: Chunhua Li
      • 14:00
        AliECS: a New Experiment Control System for the ALICE Experiment 15m

        The ALICE Experiment at CERN LHC (Large Hadron Collider) is undertaking a major upgrade during LHC Long Shutdown 2 in 2019-2020, which includes a new computing system called O² (Online-Offline). To ensure the efficient operation of the upgraded experiment and of its newly designed computing system, a reliable, high performance, and automated experiment control system is being developed. The ALICE Experiment Control System (AliECS) is a distributed system based on state of the art cluster management and microservices which have recently emerged in the distributed computing ecosystem. Such technologies will allow the ALICE collaboration to benefit from a vibrant and innovating open source community. This communication describes the AliECS architecture. It provides an in-depth overview of the system's components, features, and design elements, as well as its performance. It also reports on the experience with AliECS as part of ALICE Run 3 detector commissioning setups.

        Speaker: Teo Mrnjavac (CERN)
      • 14:15
        DAQExpert - the service to increase CMS data-taking efficiency 15m

        The Data Acquisition (DAQ) system of the Compact Muon Solenoid (CMS) experiment at LHC is a complex system responsible for the data readout, event building and recording of accepted events. Its proper functioning plays a critical role in the data-taking efficiency of the CMS experiment. In order to ensure high availability and recover promptly in the event of hardware or software failure of the subsystems, an expert system, the DAQ Expert, has been developed. It aims at improving the data taking efficiency, reducing the human error in the operations and minimising the on-call expert demand. Introduced in the beginning of 2017, it assists the shift crew and the system experts in recovering from operational faults, streamlining the post mortem analysis and, at the end of Run 2, triggering the fully automatic recoveries without a human intervention. DAQ Expert analyses the real-time monitoring data originating from the DAQ components and the high-level trigger updated every few seconds. It pinpoints the data flow problem and recovers it automatically or after given operator approval. We analyse the CMS downtime in the 2018 run focusing on what was improved with the introduction of automated recoveries; present challenges and design of transforming the expert knowledge to automated recovery jobs. Furthermore, we demonstrate the web-based, ReactJS interfaces that ensure an effective cooperation between the human operators in control room and the automated recovery system. We report on the operational experience with automated recoveries.

        Speaker: Maciej Szymon Gladki (University of Warsaw (PL))
      • 14:30
        ATLAS Operational Monitoring Data Archival and Visualization 15m

        The Information Service (IS) is an integral part of the Trigger and Data Acquisition (TDAQ) system of the ATLAS experiment at the Large Hadron Collider (LHC) at CERN. The IS allows online publication of operational monitoring data, and it is used by all sub-systems and sub-detectors of the experiment to constantly monitor their hardware and software components including more than 25000 applications running on more than 3000 computers. The Persistent Back-End for the ATLAS Information System (P-BEAST) service stores all raw operational monitoring data for the lifetime of the experiment and provides programming and graphical interfaces to access them including Grafana dashboards and notebooks based on the CERN SWAN platform. During the ATLAS data taking sessions (for the full LHC Run 2 period) P-BEAST acquired data at an average information update rate of 200 kHz and stored 20 TB of highly compacted and compressed data per year. This paper reports how over six years the P-BEAST became an essential piece of the experiment operations including details of the challenging requirements, the fails and successes of the various attempted implementations, the new types of monitoring data and the results of the time-series database technologies evaluations for the improvements during next LHC Run 3.

        Speaker: Igor Soloviev (University of California Irvine (US))
      • 14:45
        Data quality monitors of vertex detectors at the start of the Belle II experiment 15m

        The Belle II experiment features a substantial upgrade of the Belle detector and will operate at the SuperKEKB energy-asymmetric $e^+ e^-$ collider at KEK in Tuskuba, Japan. The accelerator successfully completed the first phase of commissioning in 2016 and the Belle II detector saw its first electron-positron collisions in April 2018. Belle II features a newly designed silicon vertex detector based on double-sided strip and DEPFET pixel detectors. A subset of the vertex detector was operated in 2018 to determine background conditions (Phase 2 operation); installation of the full detector was completed early in 2019 and the experiment starts full data taking.
        This talk will report on the final arrangement of the silicon vertex detector part of Belle II with focus on on-line and off-line monitoring of detector conditions and data quality, design and use of diagnostic and reference plots, and integration with the software framework of Belle II. Data quality monitoring plots will be discussed with a focus on simulation and acquired cosmic and collision data.

        Speakers: Peter Kodys (Charles University), Peter Kodys (Charles University (CZ))
      • 15:00
        Scalable monitoring data processing for the LHCb software trigger 15m

        The LHCb high level trigger (HLT) is split in two stages. HLT1 is synchronous with collisions delivered by the LHC and writes its output to a local disk buffer, which is asynchronously processed by HLT2. Efficient monitoring of the data being processed by the application is crucial to promptly diagnose detector or software problems. HLT2 consists of approximately 50000 processes and 4000 histograms are produced by each process. This results in 200 million histograms that need to be aggregated for each of up to a hundred data taking intervals that are being processed simultaneously. This paper presents the multi-level hierarchical architecture of the monitoring infrastructure put in place to achieve this. Network bandwidth is minimised by sending histogram increments and only exchanging metadata when necessary, using a custom lightweight protocol based on boost::serialize. The transport layer is implemented with ZeroMQ, which supports IPC and TCP communication, queue handling, asynchronous request/response and multipart messages. The persistent storage to ROOT is parallelized in order to cope with data arriving from a hundred of data taking intervals being processed simultaneously by HLT2. The performance and the scalability of the current system are presented. We demonstrate the feasibility of such an approach for the HLT1 use case, where real-time feedback and reliability of the infrastructure are crucial. In addition, a prototype of a high-level transport layer based on the stream-processing platform Apache Kafka is shown, which has several advantages over the lower-level ZeroMQ solution.

        Speaker: Stefano Petrucci (University of Edinburgh, CERN)
      • 15:15
        The ALICE data quality control system 15m

        The ALICE Experiment at CERN LHC (Large Hadron Collider) is undertaking a major upgrade during LHC Long Shutdown 2 in 2019-2020, which includes a new computing system called O² (Online-Offline). The raw data input from the ALICE detectors will then increase a hundredfold, up to 3.4 TB/s. In order to cope with such a large amount of data, a new online-offline computing system, called O2, will be deployed.

        One of the key software components of the O2 system will be the data Quality Control (QC) that replaces the existing online Data Quality Monitoring and offline Quality Assurance. It involves the gathering, the analysis by user-defined algorithms and the visualization of monitored data, in both the synchronous and asynchronous parts of the O2 system.

        This paper presents the architecture and design, as well as the latest and upcoming features, of the ALICE O2 QC. In particular, we review the challenges we faced developing and scaling the object merging software, the trending and correlation infrastructure and the repository management. We also discuss the ongoing adoption of this tool amongst the ALICE collaboration and the measures taken to develop, in synergy with their respective teams, efficient monitoring modules for the detectors.

        Speaker: Piotr Konopka (CERN, AGH University of Science and Technology (PL))
    • 14:00 15:30
      Track 2 – Offline Computing: ML and generative simulation Riverbank R6

      Riverbank R6

      Adelaide Convention Centre

      Convener: Chiara Ilaria Rovelli (Sapienza Universita e INFN, Roma I (IT))
      • 14:00
        Fast Simulations at LHCb 15m

        The LHCb detector at the LHC is a single forward arm spectrometer dedicated to the study of $b-$ and $c-$ hadron states. During Run 1 and 2, the LHCb experiment has collected a total of 9 fb$^{-1}$ of data, corresponding to the largest charmed hadron dataset in the world and providing unparalleled datatests for studies of CP violation in the $B$ system, hadron spectroscopy and rare decays, not to mention heavy ion and fixed target datasets. The LHCb detector is currently undergoing an upgrade to nearly all parts of the detector to cope with the increased luminosity of Run 3 and beyond. Simulation for the analyses of such datasets is paramount, but is prohibitively slow in generation and reconstruction due to the sheer number of simulated decays needed to match the collected datasets. In this talk, we explore the suite of fast simulations which LHCb has employed to meet the needs of the Run 3 and beyond, including the reuse of the underlying event and parameterized simulations, and the possibility of porting the framework to multithreaded environments.

        Speaker: Adam Davis (University of Manchester (GB))
      • 14:15
        Fast simulation methods in ATLAS: from classical to generative models 15m

        The ATLAS physics program relies on very large samples of GEANT4 simulated events, which provide a highly detailed and accurate simulation of the ATLAS detector. However, this accuracy comes with a high price in CPU, and the sensitivity of many physics analyses is already limited by the available Monte Carlo statistics and will be even more so in the future. Therefore, sophisticated fast simulation tools are developed. In Run-3 we aim to replace the calorimeter shower simulation for most samples with a new parametrized description of longitudinal and lateral energy deposits, including machine learning approaches, to achieve a fast and accurate description. Looking further ahead, prototypes are being developed using cutting edge machine learning approaches to learn the appropriate calorimeter response, which are expected to improve modeling of correlations within showers. Two different approaches, using Variational Auto-Encoders (VAEs) or Generative Adversarial Networks (GANs), are trained to model the shower simulation. Additional fast simulation tools will replace the inner detector simulation, as well as digitization and reconstruction algorithms, achieving up to two orders of magnitude improvement in speed. In this talk, we will describe the new tools for fast production of simulated events and an exploratory analysis of the deep learning methods.

        Speaker: Johnny Raine (Universite de Geneve (CH))
      • 14:30
        Learning high-level structures in HEP data with novel Deep Auto-Regressive Networks for Fast Simulation 15m

        In High Energy Physics, simulation activity is a key element for theoretical models evaluation and detector design choices. The increase in the luminosity of particle accelerators leads to a higher computational cost when dealing with the orders of magnitude increase in collected data. Thus, novel methods for speeding up simulation procedures (FastSimulation tools) are being developed with the help of Deep Learning. For this task, unsupervised learning is performed based on a given training HEP dataset with generative models employed to render samples from the same distribution.

        A novel Deep Learning architecture is proposed in this research based on autoregressive connections to model the simulation output by decomposing the event distribution as a product of conditionals. The aim is for the network to be able to capture nonlinear, long-range correlations and input varying dependencies with tractable, explicit probability densities. The following research report analyses the benefits of employing autoregressive models in comparison with previously proposed models and their ability for generalisation in the attempt of fitting multiple data distributions. The training dataset contains different simplified calorimeters simulations obtained with the Geant4 toolkit (such as: PbWO4, W/Si). Finally, testing procedures and results for network performance are developed and analysed.

        Speaker: Ioana Ifrim (CERN)
      • 14:45
        Particle shower simulation in high granularity calorimeters using 3 dimensional convolutional Generative Adversarial Networks 15m

        The future need of simulated events for the LHC experiments and their High Luminosity upgrades, is expected to increase dramatically. As a consequence, research on new fast simulation solutions, based on Deep Generative Models, is very active and initial results look promising.

        We have previously reported on a prototype that we have developed, based on 3 dimensional convolutional Generative Adversarial Network, to simulate particle showers in high-granularity calorimeters.

        As an example of future high-granularity geometries, we have chosen the electromagnetic calorimeter design developed in the context of the Linear Collider Detector studies characterised by 25 layers and 5mmx5mm cell granularity

        The training dataset is simulated using Geant4 and the DDSim framework.

        In this talk we will present improved results on a more realistic simulation of different particles (electrons and pions) characterized by variable energy and incident trajectory. Detailed validation studies, comparing our results to Geant4 Monte Carlo simulation, show very good agreement for high level physics quantities (such as energy shower shapes) and detailed calorimeter response (single cell response) over a large energy range. In particular, we will show how increasing the network representational power, introducing physics-based constraints and a transfer-learning approach to the training process improve the agreement to Geant4 results over a large energy range. Initial studies on a network optimisation based on the implementation of Genetic Algorithms will also be discussed.

        Speaker: Ricardo Brito Da Rocha (CERN)
      • 15:00
        Generative Adversarial Networks for LHCb Fast Simulation 15m

        LHCb is one of the major experiments operating at the Large Hadron Collider at CERN. The richness of the physics program and the increasing precision of the measurements in LHCb lead to the need of ever larger simulated samples. This need will increase further when the upgraded LHCb detector will start collecting data in the LHC Run 3. Given the computing resources pledged for the production of Monte Carlo simulated events in the next years, the use of fast simulation techniques will be mandatory to cope with the expected dataset size. In LHCb generative models, which are nowadays widely used for computer vision and image processing are being investigated in order to accelerate the generation of showers in the calorimeter and high-level responses of Cherenkov detector. We demonstrate that this approach provides high-fidelity results along with a significant speed increase and discuss possible implication of these results. We also present an implementation of this algorithm into LHCb simulation software and validation tests.

        Speaker: Fedor Ratnikov (Yandex School of Data Analysis (RU))
      • 15:15
        Generation of Belle II pixel detector background data with a GAN 15m

        Belle II uses a Geant4-based simulation to determine the detector response to the generated decays of interest. A realistic detector simulation requires the inclusion of noise from beam-induced backgrounds. This is accomplished by overlaying random trigger data to the simulated signal. To have statistically independent Monte-Carlo events a high number of random trigger events are desirable. However, the size of the background events, in particular the part of the pixel vertex detector (PXD), is so large that it is infeasible to record, store, and overlay the same amount as simulated signal events. Our approach to overcome the limitation of the simulation by storage resources is to use a Wasserstein generative adverserial network to generate PXD background data. A challenge is the high resolution of 250x768 pixels of in total 40 sensors with correlations between them. We will present the current status of this approach and assess its quality based on tracking performance studies.

        Speaker: Thomas Kuhr (Ludwig Maximilians Universitat (DE))
    • 14:00 15:30
      Track 3 – Middleware and Distributed Computing: HTC Sites and Related Services Riverbank R3

      Riverbank R3

      Adelaide Convention Centre

      Convener: Stefan Roiser (CERN)
      • 14:00
        Lightweight site federation for CMS support 15m

        There is a general trend in WLCG towards the federation of resources, aiming for increased simplicity, efficiency, flexibility, and availability. Although general, VO-agnostic federation of resources between two independent and autonomous resource centres may prove arduous, a considerable amount of flexibility in resource sharing can be achieved, in the context of a single WLCG VO, with a relatively simple approach. We have demonstrated this for PIC and CIEMAT, the Spanish Tier-1 and Tier-2 sites for CMS (separated by 600 Kms, ~10 ms latency), by making use of the existing CMS xrootd federation (AAA) infrastructure and profiting from the common CE/batch technology used by the two centres (HTCondor). This work describes how compute slots are shared between the two sites with transparent and efficient access to the input data irrespective of its location. This approach allows to dynamically increase the capacity of a site with idle execution slots from the remote site. Our contribution also includes measurements for diverse CMS workflows comparing performances between local and remote execution. In addition to enabling an increased flexibility in the use of the resources, this lightweight approach can be regarded as a benchmark to explore future potential scenarios, where storage resources would be concentrated in a reduced number of sites.

        Speaker: Antonio Delgado Peris (Centro de Investigaciones Energéti cas Medioambientales y Tecno)
      • 14:15
        Provision and use of GPU resources for distributed workloads via the Grid 15m

        The Queen Mary University of London WLCG Tier-2 Grid site has been providing GPU resources on the Grid since 2016. GPUs are an important modern tool to assist in data analysis. They have historically been used to accelerate computationally expensive but parallelisable workloads using frameworks such as OpenCL and CUDA. However, more recently their power in accelerating machine learning, using libraries such as TensorFlow and Coffee, has come to the fore and the demand for GPU resources has increased. Significant effort is being spent in high energy physics to investigate and use machine learning to enhance the analysis of data. GPUs may also provide part of the solution to the compute challenge of the High Luminosity LHC. The motivation for providing GPU resources via the Grid is presented. The Installation and configuration of the SLURM batch system together with Compute Elements (Cream and ARC) for use with GPUs is shown. Real world use cases are presented and the success and issues observed will be discussed. Recommendations, informed by our experiences, and our future plans will also be given.

        Speaker: Dr Daniel Peter Traynor (Queen Mary University of London (GB))
      • 14:30
        A Lightweight Job Submission Frontend and its Toolkits – HepJob 15m

        In a HEP Computing Center, at least 1 batch systems are used. As an example, at IHEP, we’ve used 3 batch systems, PBS, HTCondor and Slurm. After running PBS as local batch system for 10 years, we replaced it by HTCondor (for HTC) and Slurm (for HPC). During that period, problems came up on both user and admin sides.

        On user side, the new batch systems bring a set of new commands, which users have to learn and remember more. In particular, some users would have to use HTCondor and Slurm in the meantime. Furthermore, HTCondor and Slurm provide more functions, which means more complicated usage mode, compared to the simple PBS commands.

        On admin side, HTCondor gives more freedom to users, which becomes a problem to admins. Admins have to find the solutions for many problems: preventing users from requesting the resources they are not allowed to use, checking if the required attributes are correct, deciding which site is requested (Slurm cluster, remote sites, virtual machine sites), etc.

        For the above requirements, HepJob was developed. HepJob provides a set of simple commands to users, hep_sub, hep_q, hep_rm, etc. In the submission procedure, HepJob checks all the attributes and ensure all attributes are correct; Assigns the proper resources to users, the user and group info is obtained from the management database; Routes jobs to the targeted site; Goes through the remaining steps.

        Users can start with HepJob very easily and admins can take many prevention actions in HepJob.

        Speaker: Xiaowei Jiang (IHEP(中国科学院高能物理研究所))
      • 14:45
        Dirac-based solutions for JUNO production system 15m

        The Jiangmen Underground Neutrino Observatory (JUNO) is a multipurpose neutrino experiment, which plans to take about 2PB raw data each year starting from 2021. The experiment data plans to be stored in IHEP and have another copy in Europe (CNAF, IN2P3, JINR data centers). MC simulation tasks are expected to be arranged and operated through a distributed computing system to share efforts among data centers. The paper will present the design of the JUNO distributed computing system based on DIRAC to meet the requirements of the JUNO workflow and dataflow among data centers according to the JUNO computing model. The production system to seamlessly manage the JUNO MC simulation workflow and dataflow together is designed within the DIRAC transformation framework, in which data flows among data centers for production groups are managed based on the DIRAC data management infrastructure which uses DFC as File Catalogue, request manager to interface with FTS and transformation system to manage a bundle of files. The muon simulation with optical photon which has huge memory and CPU time problems would be the most challenging part. Therefore, multicore supports and GPU federation are considered in the system to meet this challenge. The function and performance tests to evaluate the prototype system would be also presented in the paper.

        Speaker: Xiaomei Zhang (Chinese Academy of Sciences (CN))
      • 15:00
        Scalable processing for storage events and automation of scientific workflows 15m

        Low latency, high throughput data processing in distributed environments is a key requirement of today's experiments. Storage events facilitate synchronisation with external services where the widely adopted request-response pattern does not scale because of polling as a long-running activity. We discuss the use of an event broker and stream processing platform (Apache Kafka) for storage events, with respect to automatised scientific workflows starting from file system events (dCache, GPFS) as triggers for data processing and placement.

        In a brokered delivery, the broker provides the infrastructure for routing generated events to consumer services. A client connects to the broker system and subscribes to streams of storage events which consist of data transfer records for files being uploaded, downloaded and deleted. This model is complemented by direct delivery using W3C’s Server-Sent Events (SSE) protocol. We also address the shaping of a security model, where authenticated clients are authorised to read dedicated subsets of events.

        On the compute side, the messages feed into event-driven work-flows, either user supplied software stacks or solutions based on open-source platforms like Apache Spark as analytical framework and Apache OpenWhisk for Function-as-a-Service (FaaS) and more general computational microservices. Building on cloud application templates for scalable analysis platforms, desired services can be dynamically provisioned on DESY's on-premise OpenStack cloud as well as in commercial hybrid cloud environments. Moreover, this model supports also the integration of data management tools like Rucio to address data locality e.g. to move files subsequent to processing by event-driven work-flows.

        Speaker: Michael Schuh (Deutsches Elektronen-Synchrotron DESY)
    • 14:00 15:30
      Track 4 – Data Organisation, Management and Access: distributed computing software and beyond Riverbank R8

      Riverbank R8

      Adelaide Convention Centre

      Convener: Brian Paul Bockelman (University of Nebraska Lincoln (US))
      • 14:00
        An Information Aggregation and Analytics System for ATLAS Frontier 15m

        ATLAS event processing requires access to centralized database systems where information about calibrations, detector status and data-taking conditions are stored. This processing is done on more than 150 computing sites on a world-wide computing grid which are able to access the database using the squid-Frontier system. Some processing workflows have been found which overload the Frontier system due to the Conditions data model currently in use, specifically because some of the Conditions data requests have been found to have a low caching efficiency. The underlying cause is that non-identical requests as far as the caching are actually retrieving a much smaller number of unique payloads. While ATLAS is undertaking an adiabatic transition during LS2 and Run-3 from the current COOL Conditions data model to a new data model called CREST for Run 4, it is important to identify the problematic Conditions queries with low caching efficiency and work with the detector subsystems to improve the storage of such data within the current data model. For this purpose ATLAS put together an information aggregation and analytics system. The system is based on aggregated data from the squid-Frontier logs using the Elastic Search technology. This talk describes the components of this analytics system from the server based on Flask/Celery application to the user interface and how we use Spark SQL functionalities to filter data for making plots, storing the caching efficiency results into a PostgreSQL database and finally deploying the package via a Docker container.

        Speaker: Andrea Formica (Université Paris-Saclay (FR))
      • 14:15
        High Performance Data Format for CLAS12 15m

        With increasing data volume from Nuclear Physics experiments requirements to data
        storage and access are changing. To keep up with large data sets new data formats
        are needed for efficient processing and analysis of the data. Frequently, in the
        experiments data goes through stages from data acquisition to reconstruction and
        data analysis and data is converted from one format to another causing waisted CPU
        cycles.

        In this work we present High Performance Output (HIPO) data format developed
        for CLAS12 experiment at Jefferson National Laboratory. It was designed to fit the needs
        of data acquisition and high level data analysis, to avoid data format conversions
        at different stages of data processing. The new format was designed
        to store different event topologies from reconstructed data in tagged form
        for efficient access by different analysis groups. In centralized data skimming
        applications HIPO data format significantly outperforms standard data formats
        used in Nuclear and High Energy Physics (ROOT) and industry standard formats,
        such as Apache Avro and Apache Parquet.

        Speaker: Dr Gagik Gavalian (Jefferson Lab)
      • 14:30
        XRootD 5.0.0: encryption and beyond 15m

        For almost 10 years now XRootD has been very successful at facilitating data management of LHC experiments. Being the foundation and main component of numerous solutions employed within the WLCG collaboration (like EOS and DPM), XRootD grew into one of the most important storage technologies in the High Energy Physics (HEP) community. With the latest major release (5.0.0) XRootD framework brought not only architectural improvements and functional enhancements, but also introduced a TLS based, secure version of the xroot/root data access protocol (a prerequisite for supporting access tokens).
        In this contribution we explain the xroots/roots protocol mechanics and focus on the implementation of the encryption component engineered to ensure low latencies and high throughput. We also give an overview of other developments finalized in release 5.0.0 (extended attributes support, verified close, etc.), and finally, we discuss what else is on the horizon.

        Speaker: Andrew Bohdan Hanushevsky (SLAC National Accelerator Laboratory (US))
      • 14:45
        FTS improvements for LHC Run-3 and beyond 15m

        The File Transfer Service developed at CERN and in production since 2014, has become fundamental component for LHC experiments workflows.

        Starting from the beginning of 2018 with the participation to the EU project Extreme Data Cloud (XDC) [1] and the activities carried out in the context of the DOMA TPC [2] and QoS [3] working groups, a series of new developments and improvements has been planned and performed taking also into account the requirements from the experiments.

        This talk will mainly focus on the support for OpenID Connect and the QoS integration via CDMI as output of the XDC project.

        The integration with OpenID Connect is also following the direction of the future Authentication and Authorisation Infrastructure (AAI) for WLCG experiments.
        The service scalability enhancements, the support for Xrootd and HTTP TPC and the first 'non-gridftp' transfers experiences via FTS between WLCG production sites will be also described, with an emphasis on performance comparison.
        The service enhancements are meeting the requirements for LHC Run-3 and facilitating the adoption for other HEP and non-HEP communities.

        [1] http://www.extreme-datacloud.eu/

        [2] https://twiki.cern.ch/twiki/bin/view/LCG/ThirdPartyCopy

        [3] https://twiki.cern.ch/twiki/bin/view/LCG/QoS

        Speaker: Edward Karavakis (CERN)
      • 15:00
        XRootD and Object Store: A new paradigm 15m

        The XRootD software framework is essential for data access at WLCG sites. The WLCG community is exploring and expanding XRootD functionality. This presents a particular challenge at the RAL Tier-1 as the Echo storage service is a Ceph based Erasure Coded object store. External access to Echo uses gateway machines which run GridFTP and XRootD servers. This paper will describe how third party copy, WebDav and additional authentication protocols have been added to these XRootD servers. This allows ALICE to use Echo as well as preparing for the eventual phase out of GridFTP.

        Local jobs access Echo via XCaches on every worker node. Remote jobs are increasingly accessing data via XRootD on Echo. For CMS jobs this is via their AAA service. For ATLAS, who are consolidating their storage at fewer sites, jobs are increasingly accessing data remotely. This paper describes the coninuting work to optimise both types of data access by testing different caching methods, including remotely configured XCaches (using SLATE) running on the RAL OpenStack cloud infrastructure.

        Speaker: Katy Ellis (Science and Technology Facilities Council STFC (GB))
      • 15:15
        Using WLCG data management software to support other communities 15m

        When the LHC started data taking in 2009 the data rates were unprecedented for the time and forced the WLCG community develop a range of tools for managing their data across many different sites. A decade later other science communities are finding their data requirements have grown far beyond what they can easily manage and are looking for help. The RAL Tier-1’s primary mission has always been to provide resources for the LHC experiments although 10% is set aside for non-LHC experiments. In the last 2 years the Tier-1 has received additional funding to support other scientific communities and now provides over 5 PB disk and 13PB Tape storage to them.

        RAL has run an FTS service for the LHC experiments for several years. Smaller scientific communities have used this for moving large volumes of data between sites but have frequently reported difficulties. In the past RAL also provided an LFC service for managing files stored on the Grid. The RAL Tier-1 should provide a complete data management solution for these communities and it was therefore decided to setup a Rucio instance to do this.

        In April 2018 a Rucio instance was setup at RAL which has so far been used by the AENEAS project. RAL is providing the development effort to allow a single Rucio instance to support multiple experiments. RAL also runs a DynaFed service which is providing an authentication and authorization layer in front of RAL S3 storage service.

        Speaker: Ian Collier (Science and Technology Facilities Council STFC (GB))
    • 14:00 15:30
      Track 5 – Software Development: Parallelism, accelerators and heterogenous computing Riverbank R2

      Riverbank R2

      Adelaide Convention Centre

      Convener: Dr Felice Pantaleo (CERN)
      • 14:00
        Concurrent data structures in the ATLAS offline software 15m

        In preparation for Run 3 of the LHC, the ATLAS experiment is modifying its offline software to be fully multithreaded. An important part of this is data structures that can be efficiently and safely concurrently accessed from many threads. A standard way of achieving this is through mutual exclusion; however, the overhead from this can sometimes be excessive. Fully lockless implementations are known for some data structures; however, they are typically complex, and the overhead they require can sometimes be larger than that required for locking implementations. An interesting compromise is to allow lockless access only for reading but not for writing. This often allows the data structures to be much simpler, while still giving good performance for read-mostly access patterns. This talk will show some examples of this strategy in data structures used by the ATLAS offline software. It will also give examples of synchronization strategies inspired by read-copy-update, as well as helpers for memoizing values in a multithreaded environment.

        Speaker: Scott Snyder (Brookhaven National Laboratory (US))
      • 14:15
        MarlinMT - parallelising the Marlin framework 15m

        Marlin is the event processing framework of the iLCSoft ecosystem. Originally developed
        for the ILC more than 15 years ago, it is now widely used, e.g. by CLICdp, CEPC and
        many test beam projects such as Calice, LCTPC and EU-Telescope. While Marlin is
        lightweight and flexible it was originally designed for sequential processing only.
        With MarlinMT we now evolved Marlin for parallel processing of events on multi-core
        architectures based on multi-threading. We report on the necessary developments and
        issues encountered, within Marlin as well as with the underlying LCIO EDM. A focus will be
        put on the new parallel event processing (PEP) scheduler. We conclude with first
        performance estimates, such as application speedup and memory profiling, based on parts
        of the ILD reconstruction chain that have been ported to MarlinMT.

        Speaker: Remi Ete (DESY)
      • 14:30
        Bringing heterogeneity to the CMS software framework 15m

        The advent of computing resources with co-processors, for example Graphics Processing Units (GPU) or Field-Programmable Gate Arrays (FPGA), for use cases like the CMS High-Level Trigger (HLT) or data processing at leadership-class supercomputers imposes challenges for the current data processing frameworks. These challenges include developing a model for algorithms to offload their computations on the co-processors as well as keeping the traditional CPU busy doing other work. The CMS data processing framework, CMSSW, implements multithreading using the Intel’s Threading Building Blocks (TBB) library, that utilizes tasks as concurrent units of work. In this talk we will discuss a generic mechanism to interact effectively with non-CPU resources that has been implemented in CMSSW. In addition, configuring such a heterogeneous system is challenging. In CMSSW an application is configured with a configuration file written in the Python language. The algorithm types are part of the configuration. The challenge therefore is to unify the CPU and co-processor settings while allowing their implementations to be separate. We will explain how we solved these challenges while minimizing the necessary changes to the CMSSW framework. We will also discuss on a concrete example how algorithms would offload work to NVIDIA GPUs using directly the CUDA API.

        Speaker: Oliver Gutsche (Fermi National Accelerator Lab. (US))
      • 14:45
        Heterogeneous reconstruction: combining an ARM processor with a GPU 15m

        As the mobile ecosystem has demonstrated, ARM processors and GPUs promise to deliver higher compute efficiency with a lower power consumption. One interesting platform to experiment with architectures different from a traditional x86 machine is the NVIDIA AGX Xavier SoC, that pairs a 64-bit ARM processor 8 cores with a Volta-class GPU with 512 CUDA cores. The CMS reconstruction software was ported to run on the ARM architecture, and there is an ongoing effort to rewrite some of the most time-consuming algorithms to leverage NVIDIA GPUs. In this presentation we will explore the challenges of running the CMS reconstruction software on a smll embedded device, and compare its compute performance and power consumption with those of a traditional x86 server.

        Speaker: Andrea Bocci (CERN)
      • 15:00
        GPU Usage in ATLAS Reconstruction and Analysis 15m

        With GPUs and other kinds of accelerators becoming ever more accessible, High Performance Computing Centres all around the world using them ever more, ATLAS has to find the best way of making use of such accelerators in much of its computing.
        Tests with GPUs -- mainly with CUDA -- have been performed in the past in the experiment. At that time the conclusion was that it was not advantageous for the ATLAS offline and trigger software to invest time and money into GPUs. However as the usage of accelerators has become cheaper and simpler in recent years, their re-evaluation in ATLAS's offline software is warranted.
        We will show code designs and performance results of using OpenCL, OpenACC and CUDA to perform calculations using the ATLAS offline/analysis (xAOD) Event Data Model. We compare the performance and flexibility of these different offload methods, and show how different memory management setups affect our ability to offload different types of calculations to a GPU efficiently. So that an overall throughout increase could be achieved even without highly optimising our reconstruction code specifically for GPUs.

        Speaker: Attila Krasznahorkay (CERN)
      • 15:15
        GPU-based Offline Clustering Algorithm for the CMS High Granularity Calorimeter 15m

        The future upgraded High Luminosity LHC (HL-LHC) is expected to deliver about 5 times higher instantaneous luminosity than the present LHC, producing pile-up up to 200 interactions per bunch crossing. As a part of its phase-II upgrade program, the CMS collaboration is developing a new end-cap calorimeter system, the High Granularity Calorimeter (HGCAL), featuring highly-segmented hexagonal silicon sensors (0.5-1.1 cm2) and scintillators (4-30 cm2), totalling more than 6 million channels in comparison to about 200k channels for the present CMS endcap calorimeters. For each event, the HGCAL clustering algorithm needs to reduce more than 100k hits into ~10k clusters, while keeping the fine shower structure. The same algorithm must reject pileup for further shower reconstruction. Due to the high pileup in the HL-LHC and high granularity, HGCAL clustering is confronted with an unprecedented surge of computation load. This motivates the concept of high-throughput heterogeneous computing in HGCAL offline clustering. Here we introduce a fully-parallelizable density-based clustering algorithm running on GPUs. It uses a tile-based data structure as input for a fast query of neighbouring cells and achieves an O(n) computational complexity. Within the CMS reconstruction framework, clustering on GPUs demonstrates at least a 10x throughout increase compared to current CPU-based clustering.

        Speaker: Mr Ziheng Chen (Northwestern University)
    • 14:00 15:30
      Track 6 – Physics Analysis: New Physics studies Hall G

      Hall G

      Adelaide Convention Centre

      Convener: Maurizio Pierini (CERN)
      • 14:00
        Signal versus Background Interference in $H^+\to t\bar b$ Signals for MSSM Benchmark Scenarios 15m

        In this talk I will present an investigation into sizeable interference effects between a {heavy} charged Higgs boson signal produced via $gg\to t\bar b H^-$ (+ c.c.) followed by the decay $H^-\to b\bar t$ (+ c.c.) and the irreducible background given by $gg\to t\bar t b \bar b$ topologies at the Large Hadron Collider (LHC). I will show how such effects could spoil current $H^\pm$ searches where signal and background are normally treated separately. The reason for this is that a heavy charged Higgs boson can have a large total width, in turn enabling such interferences, altogether leading to very significant alterations, both at the inclusive and exclusive level, of the yield induced by the signal alone. This therefore implies that currently established LHC searches for such wide charged Higgs bosons require modifications. This is shown quantitatively using two different benchmark configurations of the minimal realisation of Supersymmetry, wherein such $H^\pm$ states naturally exist.

        Speaker: Riley Patrick (The University of Adelaide)
      • 14:15
        GAMBIT: The Global and Modular BSM Inference Tool 15m

        GAMBIT is a modular and flexible framework for performing global fits to a wide range of theories for new physics. It includes theory and analysis calculations for direct production of new particles at the LHC, flavour physics, dark matter experiments, cosmology and precision tests, as well as an extensive library of advanced parameter-sampling algorithms. I will present the GAMBIT software framework and give a brief overview of the main physics results it has produced so far.

        Speaker: Pat Scott (The University of Queensland)
      • 14:30
        Searching for dark matter signatures in 20 years of GPS atomic clock data 15m

        Despite the overwhelming cosmological evidence for the existence of dark matter, and the considerable effort of the scientific community over decades, there is no evidence for dark matter in terrestrial experiments.
        The GPS.DM observatory uses the existing GPS constellation as a 50,000 km-aperture sensor array, analysing the satellite and terrestrial atomic clock data for exotic physics signatures. In particular, the collaboration searches for evidence of transient variations of fundamental constants correlated with the Earth’s galactic motion through the dark matter halo.
        The initial results of the search lead to an orders-of-magnitude improvement in constraints on certain models of dark matter [1].
        I will discuss the initial results and future prospects, including the method used for processing the data, and the “GPS simulator” and dark-matter signal generator we built to test to methods [2].

        [1] B. M. Roberts, G. Blewitt, C. Dailey, M. Murphy, M. Pospelov, A. Rollings, J. Sherman, W. Williams, and A. Derevianko, Nat. Commun. 8, 1195 (2017).
        [2] B. M. Roberts, G. Blewitt, C. Dailey, and A. Derevianko, Phys. Rev. D 97, 083009 (2018).

        Speaker: Benjamin Roberts (The University of Queensland)
      • 14:45
        Exploring the new physics in 2HDM 15m

        In this talk, we discuss the new physics implication in Two Higgs doublet Model (2HDM) under various experimental constraints. As part work of Gambit group, our work is to use the global fit method to constrain the parameter space, find out the hints for new physics and try to make some predictions for further studies.

        In our global fit, we include the constraints from LEP, LHC (SM-like Higgs boson search), the theoretical requirements ( Unitarity, Perturbativity, and vacuum stability), various flavour physics constraints ( radiative B Decay $B \to X_s \gamma$, rare fully leptonic B decays $B \to \mu^+\mu^-$ ,etc) and muon g-2 anomaly.

        After the 7-parameter global fit, we have a detailed study about the result, analysing individual constraints effects, finding out advantages of every constraint in constraining parameters and discovering new particles. For the Type-II 2HDM, we find that the $\lambda_2$ is sensitive to the LHC SM-lkie Higgs boson search results. Our final results will be displayed in $\tan \beta$ - $\cos(\beta-\alpha)$, $m_A$ - $\tan \beta$, which are usually considered.

        Speaker: Dr wei su (University of Adelaide)
      • 15:00
        Monte Carlo event generator with model-independent new physics effect for B->K(*)ll decays 15m

        At the high luminosity flavor factory experiments such as the Belle II
        experiment, it is expected to find the new physics effect and
        constrain the new physics models with the high statics and many
        observables. In such analysis, the global analysis of the many
        observables with the model-independent approach is important. One
        difficulty in such global analysis is that the new physics could
        affect the numerical results obtained by experiments assuming the
        Standard Model, because of the changes of the reconstructed
        kinematical distributions used in the event selection and in the
        fitting to obtain the number of signal and background events.
        Therefore, it is also important to prepare the event generator
        including the new physics effects for the Monte Carlo simulation of
        the detector response to estimate and consider the effects properly in
        the global analysis.
        In this work, we present development of the event generator of
        B->K(*)ll decays including the new physics effect in the
        model-independent way by parametrizing with the Wilson coefficients.
        We implement the decay model using the EvtGen
        [https://evtgen.hepforge.org/] framework so that it can be applicable
        in the analysis software framework of the B physics experiments. For the
        theoretical calculation of the new physics effect we consider the EOS
        [https://eos.github.io/] library and other possible calculations. We
        report the results obtained by the developed event generator and
        application in the global analysis.

        Speaker: Koji Hara (KEK)
      • 15:15
        Ways of Seeing: Finding BSM physics at the LHC 15m

        Searches for beyond-Standard Model physics at the LHC have thus far not uncovered any evidence of new particles, and this is often used to state that new particles with low mass are now excluded. Using the example of the supersymmetric partners of the electroweak sector of the Standard Model, I will present recent results from the GAMBIT collaboration that show that there is plenty of room for low mass solutions based on the LHC data. I will then present a variety of methods for designing new LHC analyses that can successfully target those solutions.

        Speaker: Martin John White (University of Adelaide (AU))
    • 14:00 15:30
      Track 7 – Facilities, Clouds and Containers: Cloud computing Riverbank R7

      Riverbank R7

      Adelaide Convention Centre

      Convener: Christoph Wissing (Deutsches Elektronen-Synchrotron (DE))
      • 14:00
        The DODAS experience on EGI Federated Cloud 15m

        The EGI Cloud Compute service offers a multi-cloud IaaS federation that brings together research clouds as a scalable computing platform for research accessible with OpenID Connect Federated Identity. The federation is not limited to single sign-on, it also introduces features to facilitate the portability of applications across providers: i) a common VM image catalogue VM image replication to ensure these images will be available at providers whenever needed; ii) a GraphQL information discovery API to understand the capacities and capabilities available at each provider; and iii) integration with orchestration tools (such as Infrastructure Manager) to abstract the federation and facilitate using heterogeneous providers. EGI also monitors the correct function of every provider and collects usage information across all the infrastructure.

        DODAS (Dynamic On Demand Analysis Service) is an open-source Platform-as-a-Service tool, which allows to deploy software applications over heterogeneous and hybrid clouds. DODAS is one of the so-called Thematic Services of the EOSC-hub project and it instantiates on-demand container-based clusters offering a high level of abstraction to users, allowing to exploit distributed cloud infrastructures with a very limited knowledge of the underlying technologies.

        This work presents a comprehensive overview of DODAS integration with EGI Cloud Federation, reporting the experience of the integration with CMS Experiment submission infrastructure system.

        Speaker: Daniele Spiga (Universita e INFN, Perugia (IT))
      • 14:15
        CloudScheduler V2: Distributed Cloud Computing in the 21st century 15m

        The cloudscheduler VM provisioning service has been running production jobs for ATLAS and Belle II for many years using commercial and private clouds in Europe, North America and Australia. Initially released in 2009, version 1 is a single Python 2 module implementing multiple threads to poll resources and jobs, and to create and destroy virtual machine. The code is difficult to scale, maintain or extend and lacks many desirable features, such as status displays, multiple user/project management, robust error analysis and handling, and time series plotting, to name just a few examples. To address these shortcomings, our team has re-engineered the cloudscheduler VM provisioning service from the ground up. The new version, dubbed cloudscheduler version 2 or CSV2, is written in Python 3 runs on any modern Linux distribution, and uses current supporting applications and libraries. The system is composed of multiple, independent Python 3 modules communicating with each other through a central MariaDB (version 10) database. It features both graphical (web browser), and command line user interfaces and supports multiples users/projects with ease. Users have the ability to manage and monitor their own cloud resources without the intervention of a system administrator. The system is scalable, extensible, and maintainable. It is also far easier to use and is more flexible than its predecessor. We present the design, highlight the development process which utilizes unit tests, and show encouraging results from our operational experience with thousands of jobs and workernodes. We also present our experience with containers for running workloads, code development and software distribution.

        Speaker: Randall Sobie (University of Victoria (CA))
      • 14:30
        Increasing interoperability for research clouds: CS3APIs for connecting sync&share storage, applications and science environments 15m

        Cloud Services for Synchronization and Sharing (CS3) have become increasing popular in the European Education and Research landscape in the last
        years. Services such as CERNBox, SWITCHdrive, CloudStor and many more have become indispensable in everyday work for scientists, engineers and in administration

        CS3 services represent an important part of the EFSS market segment (Enterprise File Sync and Share). According to the report at the last CS3 2019 Rome conference, 25 sites provide a service to the total of 395 thousand researchers and educators around the globe (in Europe and Australia, China, US, Brazil, South Africa and Russia) serving 2.7 billion files (corresponding to 11.5 PB of storage). CS3 provides easily accessible, sync&share services with intuitive and responsive user interfaces.

        Although these services are becoming popular because of their intuitive interface for sharing and synchronization of data, availability on all platforms (mobile, desktop and web) and capabilities to adapt to different user scenarios such as offline work, the commercially developed sync&share platforms are not sufficiently integrated with research services, tools and applications. This lack of integration currently forms a major bottleneck for European collaborative research communities. In addition, services as operated by several European providers who are in CS3, are currently too fragmented.

        The CS3 APIs is a set of APIs to make research clouds based on sync and share technology interoperable. The APIs are designed to decrease the burden of porting an application developed for one EFSS service to another one and also provide a standard way to connect the sync and share platform with existing and new storage repositories over a well-defined metadata control protocol. These interconnections increase the cohesion between services to create an easily-accessible and integrated science environment that facilitates research activities across institutions without having fragmented silos based on ad-hoc solutions.

        We report on our experience designing the protocol and the reference implementation (REVA), and its future evolution to reduce the fragmentation in the pan-European research network.

        Speaker: Hugo Gonzalez Labrador (CERN)
      • 14:45
        Pre-Commercial Procurement: R&D as a Service for the European Open Science Cloud 15m

        The use of commercial cloud services has gained popularity in research environments. Not only it is a flexible solution for adapting computing capacity to the researchers' needs, it also provides access to the newest functionalities on the market. In addition, most service providers offer cloud credits, enabling researchers to explore innovative architectures before procuring them at scale. Yet, the economical and contractual aspects linked to the production use of commercial clouds are often overlooked, preventing researchers to reap their full benefits.

        CERN, in collaboration with leading European research institutes, has launched several initiatives to bridge this gap. Completed in 2018, the HNSciCloud Pre-Commercial Procurement (PCP) project successfully developed a European hybrid cloud platform to pioneer the convergence and deployment of commercial cloud, high-performance computing and big-data capabilities for scientific research. Leveraging many of the lessons learned from HNSciCloud, the OCRE project - Open Clouds for Research Environments - started in January 2019 in order to accelerate commercial cloud adoption in the European research community. In parallel, the ARCHIVER PCP project - Archiving and Preservation for Research Environments - will develop hybrid and scalable solutions for archiving and long-term preservation of scientific data whilst ensuring that research groups retain stewardship of their datasets.

        With a total procurement budget exceeding €18 million, these initiatives are setting best practices for effective and sustainable procurement of commercial cloud services for research activities. These are highly relevant as, in the wider context of the European Open Science Cloud (EOSC), the engagement of commercial providers is considered fundamental to contribute to the creation of a sustainable, technologically advanced environment with open services for data management, analysis and re-use across disciplines, with transparent costing models.

        In this contribution, we will detail the outcomes of the HNSciCloud PCP project, expand on the objectives of the subsequent OCRE and ARCHIVER projects and provide a vision for the role of the private sector within the EOSC.

        Speaker: Marion Devouassoux (CERN)
      • 15:00
        Dynamic integration of distributed, Cloud-based HPC and HTC resources using JSON Web Tokens and the INDIGO IAM Service 15m

        In the last couple of years, we have been actively developing the Dynamic On-Demand Analysis Service (DODAS) as an enabling technology to deploy container-based clusters over any Cloud infrastructure with almost zero effort. The DODAS engine is driven by high-level templates written in the TOSCA language, that allows to abstract the complexity of many configuration details. DODAS is particularly suitable for harvesting opportunistic computing resources; this is why several scientific communities already integrated their computing use cases into DODAS-instantiated clusters automating the instantiation, management and federation of HTCondor batch system.
        The increasing demand, availability and utilization of HPC by and for multidisciplinary user community, often mandates the possibility to transparently integrate, manage and mix HTC and HPC resources.
        In this paper, we discuss our experience extending and using DODAS to connect HPC and HTC resources in the context of a distributed Italian regional infrastructure involving multiple sites and communities. In this use case, DODAS automatically generates HTCondor-based clusters on-demand, dynamically and transparently federating sites that may also include HPC resources managed by SLURM; DODAS allows user workloads to make opportunistic and automated use of both HPC and HTC resources, thus effectively maximizing and optimizing resource utilization.
        We also report on our experience of using and federating HTCondor batch systems exploiting the JSON Web Token capabilities introduced in recent HTCondor versions, replacing the traditional X509 certificates in the whole chain of workload authorization. In this respect we also report on how we integrated HTCondor using OAuth with the INDIGO IAM service.

        Speaker: Stefano Dal Pra (Universita e INFN, Bologna (IT))
      • 15:15
        Characterizing network paths in and out of the Clouds 15m

        Cloud computing is becoming mainstream, with funding agencies moving beyond prototyping and starting to fund production campaigns, too. An important aspect of any production computing campaign is data movement, both incoming and outgoing. And while the performance and cost of VMs is relatively well understood, the network performance and cost is not.
        We thus embarked on a network characterization campaign, documenting traceroutes, latency and throughput in various regions of Amazon AWS, Microsoft Azure and Google GCP Clouds, both between Cloud resources and major DTNs in the Pacific Research Platform, including OSG data federation caches in the network backbone, and inside the clouds themselves. We also documented the incurred cost while doing so.
        Along the way we discovered that network paths were often not what the major academic network providers thought they were, and we helped them in improving the situation, thus improving peering between academia and commercial cloud.
        In this talk we present the observed results, both during the initial test runs and the latest state of the art, as well as explain what it took to get there.

        Speaker: Igor Sfiligoi (UCSD)
    • 14:00 15:30
      Track 8 – Collaboration, Education, Training and Outreach: Education Riverbank R1

      Riverbank R1

      Adelaide Convention Centre

      Convener: Marzena Lapka (CERN)
      • 14:00
        "Physics of Data", an innovative master programme in Physics 15m

        Most of the challenges set by modern physics endeavours are related to the management, processing and analysis of massive amount of data. As stated in a recent Nature editorial (The thing about data, Nature Physics volume 13, page 717, 2017), "the rise of big data represents an opportunity for physicists. To take full advantage, however, they need a subtle but important shift in mindset". All this calls for a substantial change in the way future physicists are taught: statistics and probability, information theory, machine learning as well as scientific computing and hardware setups should be the pillars of the education of a new physics students generation. This is what an innovative master programme launched in fall 2018 by the University of Padua, "Physics of Data", aims at. This contribution summarises its actual implementation, describing the educational methods (all focused on "hands-on" activities and research projects) and reporting on the brilliant results obtained by the first enrolled students.

        Speaker: Marco Zanetti (Universita e INFN, Padova (IT))
      • 14:15
        Unplugged Computing for Children 15m

        The number of women in technical computing roles in the HEP community hovers at around 15%. At the same time there is a growing body of research to suggest that diversity, in all its forms, brings positive impact on productivity and wellbeing. These aspects are directly in line with many organisations’ values and missions, including CERN. Although proactive efforts to recruit more women in our organisations and institutes may help, the percentage of female applicants in candidate pools is similarly low and limits the potential for change. Factors influencing the career choice of girls have been identified to start as early as primary school and are closely tied to encouragement and exposure. It is the hope of various groups in the HEP community that, by intervening early, there may be a change in demographics over the years to come. During 2019, the Women in Technology Community at CERN developed two workshops for 6-9 year olds, which make the fundamental concepts of computer science accessible to young people with no prior experience and minimal assumed background knowledge. The immediate objectives were to demystify computer science, and to allow the children to meet a diverse set of role models from technical fields through our volunteer tutors. The workshops have been run for International Women’s Day and Girls in ICT Day, and a variation will be incorporated into the IT contribution to CERN’s Open Day in September 2019 (where both boys and girls will participate). We will present an overview of the statistics behind our motivation, describe the content of the workshops, results and lessons learnt and the future evolution of such activities.

        Speaker: Hannah Short (CERN)
      • 14:30
        Delivering a machine learning course on HPC resources 15m

        In recent years proficiency in data science and machine learning (ML) became one of the most requested skills for jobs in both industry and academy. Machine learning algorithms typically require large sets of data to train the models and extensive usage of computing resources both for training and inference. Especially for deep learning algorithms, training performances can be dramatically improved by exploiting Graphical Processing Units (GPU). The needed skill set for a data scientist is therefore extremely broad, and ranges from knowledge of ML models to distributed programming on heterogeneous resources. While most of the available training resources focus on ML algorithms and tools such as TensorFlow, we designed a course for doctoral students where model training is tightly coupled with underlying technologies that can be used to dynamically provision resources. Throughout the course, students have access to OCCAM, an HPC facility at the University of Torino, managed using container-based cloud-like technologies, where Computing Applications are run on Virtual Clusters deployed on top of the physical infrastructure.
        Task scheduling over OCCAM resources is managed by an orchestration layer (such as Mesos or Kubernetes), leveraging Docker containers to define and isolate the runtime environment. The Virtual Clusters developed to execute ML workflows are accessed through a web interface based on JupyterHub. When a user authenticates on the Hub, a notebook server is created as a containerized application. A set of libraries and helper functions is provided to execute a parallelized ML task by automatically deploying a Spark driver and several Spark execution nodes as Docker containers. This solution automates the delivery of the software stack required by a typical ML workflow and enables scalability by allowing the execution of ML tasks, including training, over commodity (i.e. CPUs) or high-performance (i.e. GPUs) resources distributed over different hosts across a network.

        Speaker: Federica Legger (Universita e INFN Torino (IT))
      • 14:45
        The iTHEPHY project and its software platform: enhancing remote teacher-student collaboration 15m

        iTHEPHY is an ERASMUS+ project which aims at developing innovative student-centered Deeper Learning Approaches (DPA) and Project-Based teaching and learning methodologies for HE students, contributing to increase the internationalization of physics master courses. In this talk we'll introduce the iTHEPHY project status and main goals attained, with a focus on the web-based virtual environment developed in order to support the groups of students and the teams of teachers during their DPA learning and teaching activities: the iTHEPHY DPA Platform. The iTHEPHY DPA platform will be described in detail, focusing on the methodologies and technologies which enabled us to deliver a modular, user friendly, open-source, scalable and reusable platform that can be adopted by other communities in a straightforward way. The presentation will describe the work carried out in order to integrate some well established tools like Moodle, Redmine, BigBlueButton, Rocketchat, Jitsi, Sharelatex and INDIGO-DataCloud IAM. Some aspects about containerization of services in Cloud will be also covered. Finally, some reflections about sustainability of the software platform delivered will be presented.

        Speaker: Gianluca Peco (Universita e INFN, Bologna (IT))
      • 15:00
        International Particle Physics Masterclasses - Current development to expand scope and global reach 15m

        The International Particle Physics Outreach Group (IPPOG) is a network of scientists, science educators and communication specialists working across the globe in informal science education and outreach for particle physics. IPPOG’s flagship activity is the International Particle Physics Masterclass programme, which provides secondary students with access to particle physics data using dedicated visualisation and analysis software. Students meet scientists, learn about particle physics, accelerators and detectors, perform physics measurements and search for new phenomena, then compare results in an end-of-day videoconference with other classes. The most recent of these events was held from 7 March to 16 April 2019 with thousands of students participating in 332 classes held in 239 institutes from 54 countries around the world. We report on the evolution of Masterclasses in recent years, in both physics and computing scope, as well as in global reach.

        Speaker: Farid Ould-Saada (University of Oslo (NO))
    • 14:00 15:30
      Track 9 – Exascale Science: Strategies by experiments and organizations Riverbank R4

      Riverbank R4

      Adelaide Convention Centre

      Convener: Wei Yang (SLAC National Accelerator Laboratory (US))
      • 14:00
        High Performance Computing for High Luminosity LHC 15m

        High Performance Computing (HPC) centers are the largest facilities available for science. They are centers of expertise for computing scale and local connectivity and represent unique resources. The efficient usage of HPC facilities is critical to the future success of production processing campaigns of all Large Hadron Collider (LHC) experiments. A substantial amount of R&D investigations are being performed in order to harness the power provided by such machines. HPC facilities are early adopters of heterogenous accelerated computing architectures, which represent a challenge and an opportunity. The adoption of accelerated heterogenous architectures has the potential to dramatically increase the performance of specific workflows and algorithms. In this presentation we will discuss R&D work on using alternative architectures both in collaboration with industry through CERN openlab and with the DEEP-EST project, a European consortium to build a prototype modular HPC infrastructure at the exa-scale. We will present the work on a proof-of-concept container platform and batch integration for workload submissions to access HPC testbed resources for data intensive science applications. As strategic computing resources, HPC centers are often isolated with tight network security, which represents a challenge for data delivery and access. We will close by summarizing the requirements and challenges for data access, through the Data Organization Management and Access (DOMA) project of the WLCG. Facilitating data access is critical to the adoption of HPC centers for data intensive science.

        Speaker: Maria Girone (CERN)
      • 14:15
        CMS Strategy for HPC resource exploitation 15m

        High Energy Physics (HEP) experiments will enter a new era with the start of the HL-LHC program, where computing needs required will surpass by large factors the current capacities. Looking forward to this scenario, funding agencies from participating countries are encouraging the HEP collaborations to consider the rapidly developing High Performance Computing (HPC) international infrastructures as a mean to satisfy at least a fraction of the foreseen HEP processing demands. Moreover, considering that HEP needs have been usually covered by facilities cost-optimized rather than performance-optimized, employing HPC centers would also allow access to more advanced resources. HPC systems are highly non-standard facilities, custom­-built for use cases largely different from CMS demands, namely the processing of real and simulated particle collisions which can be analyzed individually without any correlation. The utilization of these systems by HEP experiments would not trivial, as each HPC center is different, increasing the level of complexity from the CMS integration and operations perspectives. Additionally, while CMS data is residing on a distributed highly-interconnected storage infrastructure, HPC systems are in general not meant for accessing large data volumes residing outside the facility. Finally, the allocation policies to these resources is quite different from the current usage of pledged resources deployed at CMS supporting Grid sites. This contribution will report on the CMS strategy developed to make effective use of HPC resources, involving a closer collaboration between CMS and HPC centers in order to further understand and subsequently overcome the present obstacles. Progress in the necessary technical and operational adaptations being made in CMS computing will be described.

        Speaker: Antonio Perez-Calero Yzquierdo (Centro de Investigaciones Energéti cas Medioambientales y Tecno)
      • 14:30
        Modeling of the CMS HL-LHC computing system 15m

        The High-Luminosity LHC will provide an unprecedented data volume of complex collision events. The desire to keep as many of the "interesting" events for investigation by analysts implies a major increase in the scale of compute, storage and networking infrastructure required for HL-LHC experiments. An updated computing model is required to facilitate the timely publication of accurate physics results from HL-LHC data samples. This talk discusses the study of the computing requirements for CMS during the era of the HL-LHC. We will discuss how we have included requirements beyond the usual CPU, disk and tape estimates made by LHC experiments during Run 2, such as networking and tape read/write rate requirements. We will show how Run 2 monitoring data has been used to make choices towards a HL-LHC computing model. We will illustrate how changes to the computing infrastructure or analysis approach can impact total resource needs and cost. Finally, we will discuss the approach and status of the CMS process for evolving its HL-LHC computing model based on modeling and other factors.

        Speaker: David Lange (Princeton University (US))
      • 14:45
        Integrating LHCb workflows on HPC resources: status and strategies 15m

        High Performance Computing (HPC) supercomputers are expected to play an increasingly important role in HEP computing in the coming years. While HPC resources are not necessarily the optimal fit for HEP workflows, computing time at HPC centers on an opportunistic basis has already been available to the LHC experiments for some time, and it is also possible that part of the pledged computing resources will be offered as CPU time allocations at HPC centers in the future. The integration of the experiment workflows to make the most efficient use of HPC resources is therefore essential.

        This presentation will describe the work that has been necessary to integrate LHCb workflows at HPC sites. This has required addressing two types of challenges: in the distributed computing area, for efficiently submitting jobs, accessing the software stacks and transferring data files; and in the software area, for optimising software performance on hardware architectures that differ significantly from those traditionally used in HEP. The talk will cover practical experience for the deployment of Monte Carlo generation and simulation workflows at the HPC sites available to LHCb. It will also describe the work achieved on the software side to improve the performance of these applications using parallel multi-process and multi-threaded approaches.

        Speaker: Federico Stagni (CERN)
      • 15:00
        Enabling ATLAS big data processing on Piz Daint at CSCS 15m

        Predictions for requirements for the LHC computing for Run 3 and Run 4 (HL_LHC) over the course of the next 10 years show a considerable gap between required and available resources, assuming budgets will globally remain flat at best. This will require some radical changes to the computing models for the data processing of the LHC experiments. Concentrating computational resources in fewer larger and more efficient centres should increase the cost-efficiency of the operation and, thus, of the data processing. Large scale general purpose HPC centres could play a crucial role in such a model. We report on the technical challenges and solutions adopted to enable the processing of the ATLAS experiment data on the European flagship HPC Piz Daint at CSCS, now acting as a pledged WLCG Tier-2 centre. As the transition of the Tier-2 from classic to HPC resources has been finalised, we also report on performance figures over two years of production running and on efforts for a deeper integration of the HPC resource within the ATLAS computing framework at different tiers.

        Speaker: Francesco Giovanni Sciacca (Universitaet Bern (CH))
    • 15:30 16:00
      Break 30m
    • 16:00 18:10
      Plenary: Computational astrophysics / Diversity Hall G

      Hall G

      Adelaide Convention Centre

      Convener: James Zanotti (University of Adelaide)
      • 16:00
        Square Kilometer Array computing - from SDP to SRC 30m
        Speaker: Minh Huynh
      • 16:30
        Evaluating Rucio outside ATLAS: Common experiences from Belle II, CMS, DUNE, SKA, and LIGO 30m

        For many scientific projects, data management is an increasingly complicated challenge. The number of data-intensive instruments generating unprecedented volumes of data is growing and their accompanying workflows are becoming more complex. Their storage and computing resources are heterogeneous and are distributed at numerous geographical locations belonging to different administrative domains and organizations. These locations do not necessarily coincide with the places where data is produced nor where data is stored, analyzed by researchers, or archived for safe long-term storage. To fulfill these needs, the data management system Rucio has been developed to allow the high-energy physics experiment ATLAS to manage its large volumes of data in an efficient and scalable way.
        But ATLAS is not alone, and several diverse scientific projects have started evaluating, adopting, and adapting the Rucio system for their own needs. As the Rucio community has grown many improvements have been introduced, customisations have been added, and many bugs have been fixed. Additionally, new dataflows have been investigated and operational experiences have been documented. In this article we collect and compare the common successes, pitfalls, and oddities which arose in the evaluation efforts of multiple diverse experiments, and compare them with the ATLAS experience. This includes the high-energy physics experiments CMS and Belle II, the neutrino experiment DUNE, as well as the LIGO and SKA astronomical observatories.

        Speaker: Mario Lassnig (CERN)
      • 17:00
        Alert systems, from astrophysics experiments to telescopes 30m
        Speaker: Mansi Kasliwal (California Institute of Technology)
      • 17:30
        Diversity & Inclusivity 30m
        Speaker: Lyn Beazley
      • 18:00
        Diversity & Inclusivity at CHEP 2019 10m
        Speaker: Waseem Kamleh (University of Adelaide)
    • 18:20 19:50
      Social Event: Welcome Reception Foyer F

      Foyer F

      Adelaide Convention Centre

    • 09:00 10:30
      Plenary: Real-time computing / Future strategy Hall G

      Hall G

      Adelaide Convention Centre

      Convener: Xiaomei Zhang (Chinese Academy of Sciences (CN))
      • 09:00
        Real-time data analysis model at the LHC and connections to other experiments / fields 30m
        Speaker: Arantza De Oyanguren Campos (Univ. of Valencia and CSIC (ES))
      • 09:30
        ALICE continuous readout and data reduction strategy for Run3 30m

        The ALICE experiment has originally been designed as a relatively low-rate experiment, in particular given the limitations of the Time Projection Chamber (TPC) readout system using MWPCs. This will not be the case anymore for LHC Run 3 scheduled to start in 2021.
        After the LS2 upgrades, including a new silicon tracker and a GEM-based readout for the TPC, ALICE will operate at a peak Pb-Pb collision rate of 50 kHz.
        To cope with this rate at least the TPC will be operated in continuous mode and all collisions will be read out, compressed and written to permanent storage without any trigger selection.
        The First Level Processing (FLP) site will receive continuous raw data at the rate of 3.4 TB/s and send to Event Processing Nodes (EPN) 10-20 ms packets at a rate of about 640 GB/s.
        EPNs will perform the data reconstruction and compression in a quasi-streaming and will send them for archival at the rate of 100 GB/s.
        Here we present the details of this synchronous stage of ALICE data processing.

        Speaker: Ruben Shahoyan (CERN)
      • 10:00
        Future of software and computing in the scope of the European Strategy for Particle Physics 30m
        Speaker: Xinchou Lou (Chinese Academy of Sciences (CN))
    • 10:30 11:00
      Break 30m
    • 11:00 12:30
      Track 1 – Online and Real-time Computing: Trigger farms and networks Riverbank R5

      Riverbank R5

      Adelaide Convention Centre

      Convener: Steven Schramm (Universite de Geneve (CH))
      • 11:00
        Assessment of the ALICE O2 readout servers 15m

        The ALICE Experiment at CERN LHC (Large Hadron Collider) is undertaking a major upgrade during LHC Long Shutdown 2 in 2019-2020. The raw data input from the detector will then increase a hundredfold, up to 3.4 TB/s. In order to cope with such a large throughput, a new Online-Offline computing system, called O2, will be deployed.

        The FLP servers (First Layer Processor) are the readout nodes hosting the CRU (Common Readout Unit) cards in charge of transferring the data from the detector links to the computer memory. The data then flows through a chain of software components until it is shipped over network to the processing nodes.

        In order to select a suitable platform for the FLP, it is essential that the hardware and the software are tested together. Each candidate server is therefore equipped with multiple readout cards (CRU), one InfiniBand 100G Host Channel Adapter, and the O2 readout software suite. A series of tests are then run to ensure the readout system is stable and fulfils the data throughput requirement of 42Gb/s (highest data rate in output of the FLP equipped with 3 CRUs).

        This paper presents the software and firmware features developed to evaluate and validate different candidates for the FLP servers. In particular we describe the data flow from the CRU firmware generating data, up to the network card where the buffers are sent over the network using RDMA. We also discuss the testing procedure and the results collected on different servers.

        Speaker: Filippo Costa (CERN)
      • 11:15
        CMS Event-Builder Performance on State-of-the-Art Hardware 15m

        We report on performance measurements and optimizations of the event-builder software for the CMS experiment at the CERN Large Hadron Collider (LHC). The CMS event builder collects event fragments from several hundred sources. It assembles them into complete events that are then handed to the High-Level Trigger (HLT) processes running on O(1000) computers. We use a test system with 16 dual-socket Skylake-based computers interconnected with 100 Gbps Infiniband and Ethernet networks. The main challenge is the demanding message rate and memory performance required of the event-builder node to fully exploit the network capabilities. Each event-builder node has to process several TCP/IP streams from the detector backends at an aggregated bandwidth of 100 Gbps, distribute event fragments to other event-builder nodes at the fist level trigger rate of 100 kHz, verify and build complete events using fragments received from all other nodes, and finally make the complete events available to the HLT processors. The achievable performance on today's hardware and different system architectures is described. Furthermore, we compare native Infiniband with RoCE (RDMA over Converged Ethernet). We discuss the required optimizations and highlight some of the problems encountered. We conclude with an outlook on the baseline CMS event-builder design for the LHC Run 3 starting in 2021.

        Speaker: Remi Mommsen (Fermi National Accelerator Lab. (US))
      • 11:30
        Results from the CBM mini-FLES Online Computing Cluster Demonstrator 15m

        The Compressed Baryonic Matter (CBM) experiment is currently under construction at the GSI/FAIR accelerator facility in Darmstadt, Germany. In CBM, all event selection is performed in a large online processing system, the “First-level Event Selector” (FLES). The data are received from the self-triggered detectors at an input-stage computer farm designed for a data rate of 1 TByte/s. The distributed input interface will be realized using custom FPGA-based PCIe add-on cards, which preprocess and index the incoming data streams. The data is then transferred to an online processing cluster of several hundred nodes, which will be located in the shared Green-IT data center on campus.

        Employing a time-based container data format to decontextualize the time-stamped signal messages from the detectors, data segments of specific time intervals can be distributed on the farm and processed independently. Timeslice building, the continuous process of collecting the data of a time interval simultaneously from all detectors, places a high load on the network and requires careful scheduling and management. Optimizing the design of the online data management includes minimizing copy operations of data in memory, using DMA/RDMA wherever possible, reducing data interdependencies, and employing large memory buffers to limit the critical network transaction rate.

        As a demonstrator for the future FLES system, the mini-FLES system has been set up and is currently in operation at the GSI/FAIR facility. Designed as a vertical slice of the full system, it contains a fraction of all foreseen components. It is used to verify the developed hardware and software architecture and includes an initial version of a FLES control system. As part of the mini-CBM experiment of FAIR Phase-0, it is also the central data acquisition and online monitoring system of a multi-detector setup for physics data taking. This presentation will give an overview of the mini-FLES system of the CBM experiment and discuss its performance. The presented material includes latest results from operation in several recent mini-CBM campaigns at the GSI/FAIR SIS18.

        Speaker: Jan de Cuveland (Johann-Wolfgang-Goethe Univ. (DE))
      • 11:45
        Performance of Belle II High Level Trigger in the First Physics Run 15m

        The Belle II experiment is a new generation B-factory experiment at KEK in Japan aiming at the search for New Physics in a huge sample of B-meson dacays. The commissioning of accelerator and detector for the first physics run has been started from March this year. The Belle II High Level Trigger (HLT) is fully
        working in the beam run. The HLT is now operated with 1600 cores clusterized in 5 units of 16 processing servers, which is the 1/4 of full configuration.

        In each unit, the event-by-event basis parallel processing is implemented using the IPC-based ring buffers with the event transport over the network socket connection. The load balancing is automatically ensured by the ring buffers. In each processing server with 20 cores, the parallel processing is implemented utilizing the multi-process approach to run the same code for different events without taking a special care. The copy-on-write fork of processes efficiently reduces the memory consumption.

        The event selection is done using the same offline code in two steps. The first is the track finding and the calorimeter clustering, and the rough event selection is performed to discard off-vertex background events efficiently. The full event reconstruction is performed for the selected events and they are classified in multiple categories. Only the events in the categories of interest are finally sent out to the storage. The live data quality monitoring is also performed on HLT.

        For the selected events, the reconstructed tracks are extrapolated to the surface of pixel detector(PXD) and lively fed back to the readout electronics for the real time data reduction by sending only the associated hits.

        Currently the accelerator study is still on-going to increase the luminosity and the physics data taking is being performed by sharing the time. During the data taking, sometimes the background rate becomes high and the L1 trigger rate reaches close to 10kHz, which is the 1/3 of maximum design rate. In this condition, the performance of Belle II HLT is discussed with the detailed report on various kinds of troubles and their fixes.

        Speaker: Prof. Ryosuke Itoh (KEK)
      • 12:00
        Design of the data distribution network for the ALICE Online-Offline (O2) facility 15m

        ALICE (A Large Ion Collider Experiment), one of the large LHC experiments, is currently undergoing a significant upgrade. Increase in data rates planned for LHC Run3, together with triggerless continuous readout operation, requires a new type of networking and data processing infrastructure.

        The new ALICE O2 (online-offline) computing facility consists of two types of nodes: First Level Processors (FLP): containing a custom PCIe cards to receive data from detectors, and Event Processing Nodes (EPN): compute dense nodes equipped with GPGPUs for fast online data compression. FLPs first buffer the detector data for a time interval into SubTimeFrame (STF) objects. A TimeFrame then aggregates all corresponding STFs from each FLP into the TimeFrame (TF) object, located on a designated EPN node where it can be processed. The data distribution network connects FLP and EPN nodes, enabling efficient TimeFrame aggregation and providing a high quality of service.

        We present design details of the data distribution network tailored to the requirements of the ALICE O2 facility based on the InfiniBand HDR technology. Further, we will show a scheduling algorithm for TimeFrame distribution from FLP to EPN nodes, which evenly utilizes all available processing capacity and avoids creating long-term network congestion.

        Speaker: Gvozden Neskovic (Johann-Wolfgang-Goethe Univ. (DE))
      • 12:15
        Network simulation of a 40 MHz event building system for the LHCb experiment 15m

        The LHCb experiment will be upgraded in 2021 and a new trigger-less readout system will be implemented. In the upgraded system, both event building (EB) and event selection will be performed in software for every collision produced in every bunch-crossing of the LHC. In order to transport the full data rate of 32 Tb/s we will use state of the art off-the-shelf network technologies, e.g. InfiniBand EDR.
        The full event building system will require around 500 nodes interconnected together via a non blocking topology, because of the size of the system it very difficult to test at production scale, before the actual procurement. We resort therefore to network simulations as a powerful tool for finding the optimal configuration. We developed an accurate low level description of an InfiniBand based network with event building like traffic.
        We will present a full scale simulation of a possible implementation of the LHCb EB network.

        Speaker: Flavio Pisani (Universita e INFN, Bologna (IT))
    • 11:00 12:30
      Track 2 – Offline Computing: G4 and simulation frameworks Riverbank R6

      Riverbank R6

      Adelaide Convention Centre

      Convener: Chris Pinkenburg
      • 11:00
        Geant4 electromagnetic physics progress 15m

        The Geant4 electromagnetic (EM) physics sub-packages is an important component of LHC experiment simulations. During long shutdown 2 for LHC these packages are under intensive development and in this work we report a progress for the new Geant4 version 10.6. These developments includes modifications allowing speed-up computations for EM physics, improve EM models, extend set for models, and extend validations of EM physics. Results of EM tests and benchmarks will be discussed in details.

        Speaker: Ivana Hrivnacova (Centre National de la Recherche Scientifique (FR))
      • 11:15
        Dealing with High Background Rates in Simulations of the STAR Heavy Flavor Tracker 15m

        The STAR Heavy Flavor Tracker (HFT) has enabled a rich physics program, providing important insights into heavy quark behavior in heavy ion collisions. Acquiring data during the 2014 through 2016 runs at the Relativistic Heavy Ion Collider (RHIC), the HFT consisted of four layers of precision silicon sensors. Used in concert with the Time Projection Chamber (TPC), the HFT enables the reconstruction and topological identification of tracks arising from charmed hadron decays. The ultimate understanding of the detector efficiency and resolution demands large quantities of high quality simulations, accounting for the precise alignment of sensors, and the detailed response of the detectors and electronics to the incident tracks. The background environment presented additional challenges, as simulating the significant rates from pileup events accumulated during the long integration times of the tracking detectors could have quickly exceeded the available computational resources, and the relative contributions from different sources was unknown. STAR has long addressed these issues by embedding simulations into background events directly sampled during data taking at the experiment. This technique has the advantage of providing a completely realistic picture of the dynamic background environment while introducing minimal additional computational overhead compared to simulation of the primary collision alone, thus scaling to any luminosity. We will discuss how STAR has applied this technique to the simulation of the HFT, and will show how the careful consideration of misalignment of precision detectors and calibration uncertainties results in the detailed reproduction of basic observables, such as track projection to the primary vertex. We will further summarize the experience and lessons learned in applying these techniques to heavy-flavor simulations and discuss recent results.

        Speaker: Jason Webb (Brookhaven National Lab)
      • 11:30
        A VecGeom navigator plugin for Geant4 15m

        VecGeom is a geometry modeller library with hit-detection features as needed by particle detector simulation at the LHC and beyond. It was incubated by a Geant-R&D initiative and the motivation to combine the code of Geant4 and ROOT/TGeo into a single, better maintainable piece of software within the EU-AIDA program.

        So far, VecGeom is mainly used by LHC experiments as a geometry primitive library called from Geant4, where it has had very positive impact on CPU time due to its faster algorithms for complex primitives.

        In this contribution, we turn to a discussion of how VecGeom can be used as the navigating library in Geant4 in order to benefit from both the fast geometry primitives as well as its vectorized navigation module. We investigate whether this integration provides the speed improvements expected, on top of that obtained from geometry primitives. We discuss and benchmark the application of such a VecGeom-navigator plugin to Geant4 for the use-case of ALICE and show paths towards usage for other experiments.

        Lastly, an update on the general developments of VecGeom is given.
        This includes a review of how developments in VecGeom can further benefit from interfacing with external ray-tracing kernels such as Intel-Embree.

        Speaker: Sandro Christian Wenzel (CERN)
      • 11:45
        Integration and Performance of New Technologies in the CMS Simulation 15m

        The HL-LHC and the corresponding detector upgrades for the CMS experiment will present extreme challenges for the full simulation. In particular, increased precision in models of physics processes may be required for accurate reproduction of particle shower measurements from the upcoming High Granularity Calorimeter. The CPU performance impacts of several proposed physics models will be discussed. There are several ongoing research and development efforts to make efficient use of new computing architectures and high performance computing systems for simulation. The integration of these new R&D products in the CMS software framework and corresponding CPU performance improvements will be presented.

        Speaker: Kevin Pedro (Fermi National Accelerator Lab. (US))
      • 12:00
        Status of JUNO simulation software 15m

        The JUNO (Jiangmen Underground Neutrino Observatory) experiment is a multi-purpose neutrino experiment designed to determine the neutrino mass hierarchy and precisely measure oscillation parameters. It is composed of a 20kton liquid scintillator central detector equipped with 18000 20’’ PMTs and 25000 3’’ PMTs, a water pool with 2000 20’’ PMTs, and a top tracker. Monte-Carlo simulation is a fundamental tool for optimizing the detector design, tuning reconstruction algorithms, and performing physics study. The status of the JUNO simulation software will be presented, including generator interface, detector geometry, physics processes, MC truth, pull mode electronic simulation and background mixing. This contribution will also present the latest update of JUNO simulation software, including Geant4 upgraded from 9.4 to 10.4, and their performance comparison. Previous electronic simulation algorithm can only work for central detector, a new electronic simulation package is designed to enable joint simulation of all sub-detectors by using the Task/sub-Task/Algorithm/Tool scheme provided by SNiPER framework. The full simulation of optical photons in large liquid scintillator is CPU intensive, especially for cosmic muons, atmospheric neutrinos and proton decay events. For proton decay, users are only interested in the proton decay events with energy deposition between 100MeV and 700MeV, number of Michel electrons larger than 0, and the energy of Michel electron larger than 10MeV. Only 10% of the simulated proton decay events meet these requirements. We made some improvements on simulation procedure to enable doing full optical photon simulation only on some pre-selected events and reduce a lot of computing resources. A pre-simulation without optical photon simulation is carried out firstly, with all the Geant4 steps and other necessary MC truth information saved. Then a pre-selection based on MC truth information can determine which event need to be full simulated with optical photon processes activated and the G4Steps as input. This simulation procedure and relative interfaces can also be used for MPI or GPU based muon events full simulation.

        Speaker: Dr Ziyan Deng (Institute of High Energy Physics)
      • 12:15
        System simulations for the ALICE ITS detector upgrade 15m

        The ALICE experiment at the CERN LHC will feature several upgrades for run 3, one of which is a new inner tracking system (ITS). The ITS upgrade is currently under development and commissioning. The new ITS will be installed during the ongoing long shutdown 2.

        The specification for the ITS upgrade calls for event rates of up to 100 kHz for Pb-Pb, and 400 kHz pp, which is two orders of magnitude higher than the existing system. The seven layers of ALPIDE pixel sensor chips significantly improve tracking with a total of 24120 pixel chips. This is a vast improvement over the existing inner tracker with six layers, of which only the two innermost layers were pixel sensors.

        A number of factors will have an impact on the performance and readout efficiency of the upgraded ITS in run 3. While these factors are not limited to operating conditions such as run type and event rates, there are also a number of sensor configuration parameters that will have an effect. For instance the strobe length and the choice of sensor operating mode; triggered or continuous.

        To that end we have developed a simplified simulation model of the readout hardware in the ALPIDE and ITS, using the SystemC library for system level modeling in C++. This simulation model is three orders of magnitude faster than a normal HDL simulation of the chip, and facilitates simulations of an increased number of events for a large portion of the detector.

        In this paper we present simulation results where we have been able to quantify detector performance under different running conditions. The results are used for system configuration as well as ongoing development of the readout electronics.

        Speaker: Simon Voigt Nesbo (Western Norway University of Applied Sciences (NO))
    • 11:00 12:30
      Track 3 – Middleware and Distributed Computing: HPCs and Related Services Riverbank R3

      Riverbank R3

      Adelaide Convention Centre

      Convener: James Letts (Univ. of California San Diego (US))
      • 11:00
        Harnessing the power of supercomputers using the PanDA Pilot 2 in the ATLAS Experiment 15m

        The unprecedented computing resource needs of the ATLAS experiment have motivated the Collaboration to become a leader in exploiting High Performance Computers (HPCs). To meet the requirements of HPCs, the PanDA system has been equipped with two new components; Pilot 2 and Harvester, that were designed with HPCs in mind. While Harvester is a resource-facing service which provides resource provisioning and workload shaping, Pilot 2 is responsible for payload execution on the resource.
        The presentation focuses on Pilot 2, which is a complete rewrite of the original PanDA Pilot used by ATLAS and other experiments for well over a decade. Pilot 2 has a flexible and adaptive design that allows for plugins to be defined with streamlined workflows. In particular, it has plugins for specific hardware infrastructures (HPC/GPU clusters) as well as for dedicated workflows defined by the needs of an experiment.
        Examples of dedicated HPC workflows are discussed in which the Pilot either uses an MPI application for processing fine-grained event level service under the control of the Harvester service or acts like an MPI application itself and runs a set of job in an assemble.
        In addition to describing the technical details of these workflows, results are shown from its deployment on Cori (NERSC), Theta (ALCF), Titan and Summit (OLCF).

        Speaker: Paul Nilsson (Brookhaven National Laboratory (US))
      • 11:15
        IceProd Supercomputer Mode: How IceCube Production Runs on Firewalled Clusters 15m

        For the past several years, IceCube has embraced a central, global overlay grid of HTCondor glideins to run jobs. With guaranteed network connectivity, the jobs themselves transferred data files, software, logs, and status messages. Then we were given access to a supercomputer, with no worker node internet access. As the push towards HPC increased, we had access to several of these machines, but no easy way to use them. So we went back to the basics of running production jobs, staging data in and out and running offline jobs on the local queue. But we made sure it still integrated directly with our dataset management and file metadata systems, to not lose everything we had gained in recent years.

        Speaker: David Schultz (University of Wisconsin-Madison)
      • 11:30
        Scheduling, deploying and monitoring 100 million tasks 15m

        The SKA will enable the production of full polarisation spectral line cubes at a very high spatial and spectral resolution. Performing a back-of-the-evelope estimate gives you the incredible amount of around 75-100 million tasks to run in parallel to perform a state-of-the-art faceting algorithm (assuming that it would spawn off just one task per facet, which is not the case). This simple estimate formed the basis of the development of a prototype, which had scalability as THE primary requirement. In this talk I will present the current status of the DALiuGE system, including some really exciting computer science research.

        Speaker: Prof. Andreas Wicenec (International Centre of Radio Astronomy Research)
      • 11:45
        Managing the ATLAS Grid through Harvester 15m

        ATLAS Computing Management has identified the migration of all resources to Harvester, PanDA’s new workload submission engine, as a critical milestone for Run 3 and 4. This contribution will focus on the Grid migration to Harvester.
        We have built a redundant architecture based on CERN IT’s common offerings (e.g. Openstack Virtual Machines and Database on Demand) to run the necessary Harvester and HTCondor services, capable of sustaining the load of O(1M) workers on the grid per day.
        We have reviewed the ATLAS Grid region by region and moved as much possible away from blind worker submission, where multiple queues (e.g. single core, multi core, high memory) compete for resources on a site. Instead we have migrated towards more intelligent models that use information and priorities from the central PanDA workload management system and stream the right number of workers of each category to a unified queue while keeping late binding to the jobs.
        We will also describe our enhanced monitoring and analytics framework. Worker and job information is synchronized with minimal delays to a CERN IT provided Elastic Search repository, where we can interact with dashboards to follow submission progress, discover site issues (e.g. broken Compute Elements) or spot empty workers.
        The result is a much more efficient usage of the Grid resources with smart, built-in monitoring of resources.

        Speaker: Fernando Harald Barreiro Megino (University of Texas at Arlington)
      • 12:00
        Nordugrid ARC cache: Efficiency gains on HPC and cloud resources 15m

        The WLCG is today comprised of a range of different types of resources such as cloud centers, large and small HPC centers, volunteer computing as well as the traditional grid resources. The Nordic Tier 1 (NT1) is a WLCG computing infrastructure distributed over the Nordic countries. The NT1 deploys the Nordugrid ARC CE, which is non-intrusive and lightweight, originally developed to cater for HPC centers where no middleware could be installed on the compute nodes. The NT1 runs ARC in the Nordugrid mode which contrary to the Pilot mode leaves jobs data transfers up to ARC. ARCs data transfer capabilities together with the ARC cache are the most important features of ARC.

        HPCs are getting increased interest within the WLCG, but so are cloud resources. With the ARC CE as an edge service to the cloud or HPC resource, all data transfers required by a job are downloaded by data transfer nodes on the edge of the cluster before the job starts running on the compute node. This ensures a highly efficient use of the compute nodes CPUs, as the job starts immediately after reaching the compute node compared to the traditional pilot model where the pilot job on the compute node is responsible for fetching the data. In addition, the ARC cache gives a possible several-fold gain if more jobs need the same data. ARCs data handling capabilities ensures very efficient data access to the jobs, and even better for HPC centers with its fast interconnects.

        In this presentation we will describe the Nordugrid model with the ARC-CE as an edge service to an HPC or cloud resource and show the gain in efficiency this model provides compared to the pilot model.

        Speaker: Maiken Pedersen (University of Oslo (NO))
      • 12:15
        Reusing distributed computing software and patterns for midscale collaborative science 15m

        Many of the challenges faced by the LHC experiments (aggregation of distributed computing resources, management of data across multiple storage facilities, integration of experiment-specific workflow management tools across multiple grid services) are similarly experienced by "midscale" high energy physics and astrophysics experiments, particularly as their data set volumes are increasing at comparable rates. Often these (international, multi-institution) collaborations have outgrown the computing resources offered by their home laboratories, or the capacities of any single member institution. Unlike the LHC experiments, however, these collaborations often lack the manpower required to build, integrate and operate the systems required to meet their scale. In the Open Science Grid, we have organized a team designed to support collaborative science organizations re-use proven software and patterns in distributed processing and data management, often but not restricted to software developed for the LHC. Examples are re-use of the Rucio and FTS3 software for reliable data transfer and management, XRootD for data access and caching, Ceph for large scale pre-processing storage, and Pegasus for workflow management across heterogeneous resources. We summarize experience with the VERITAS gamma ray observatory, the South Pole Telescope (CMB detector), and the XENON dark matter search experiment.

        Speakers: Benedikt Riedel (University of Wisconsin-Madison), Benedikt Riedel (University of Wisconsin-Madison)
    • 11:00 12:30
      Track 4 – Data Organisation, Management and Access: Data transfer, data access and storage QoS Riverbank R8

      Riverbank R8

      Adelaide Convention Centre

      Convener: Tigran Mkrtchyan (DESY)
      • 11:00
        Modernizing Third-Party-Copy Transfers in WLCG 15m

        The “Third Party Copy” (TPC) Working Group in the WLCG’s “Data Organization, Management, and Access” (DOMA) activity was proposed during a CHEP 2018 Birds of a Feather session in order to help organize the work toward developing alternatives to the GridFTP protocol. Alternate protocols enable the community to diversify; explore new approaches such as alternate authorization mechanisms; and reduce the risk due to the retirement of the Globus Toolkit, which provides a commonly used GridFTP protocol implementation.

        Two alternatives were proposed to the TPC group for investigation: WebDAV and XRootD. Each approach has multiple implementations, allowing us to demonstrate interoperability between distinct storage systems. As the working group took as a mandate the development of alternatives - and not to select a single protocol - we have put together a program of work allowing both to flourish. This includes community infrastructure such as documentation pointers, email lists, or biweekly meetings, as well as continuous interoperability testing involving production & test endpoints, deployment recipes, scale testing, and debugging assistance.

        Each major storage system utilized by WLCG sites now has at least one functional non-GridFTP protocol for performing third-party-copy. The working group is focusing on including a wider set of sites and helping sites deploy more production endpoints. We are interacting with WLCG VOs to perform production data transfers using WebDAV or XRootD at the participating sites with the objective that all sites deploy at least one of these alternative protocols.

        Speaker: Alessandra Forti (University of Manchester (GB))
      • 11:15
        Third-party transfers in WLCG using HTTP 15m

        Since its earliest days, the Worldwide LHC Computational Grid (WLCG) has relied on GridFTP to transfer data between sites. The announcement that Globus is dropping support of its open source Globus Toolkit (GT), which forms the basis for several FTP client and servers, has created an opportunity to reevaluate the use of FTP. HTTP-TPC, an extension to HTTP compatible with WebDAV, has arisen as a strong contender for an alternative approach.

        In this paper, we describe the HTTP-TPC protocol itself, along with the current status of its support in different implementations, and the interoperability testing done within the WLCG DOMA working group’s TPC activity. This protocol also provides the first real use-case for token-based authorisation. We will demonstrate the benefits of such authorisation by showing how it allows HTTP-TPC to support new technologies (such as OAuth, OpenID Connect, Macaroons and SciTokens) without changing the protocol. We will also discuss the next steps for HTTP-TPC, improving documentation and plans to use the protocol for WLCG transfers.

        Speaker: Brian Paul Bockelman (University of Nebraska Lincoln (US))
      • 11:30
        Xrootd Third Party Copy for the WLCG and HL-LHC 15m

        A Third Party Copy (TPC) has existed in the pure XRootD storage environment for many years. However using XRootD TPC in the WLCG environment presents additional challenges due to the diversity of the storage systems involved such as EOS, dCache, DPM and ECHO, requiring that we carefully navigate the unique constraints imposed by these storage systems and their site-specific environments through customized configuration and software development. To support multi-tenant setups seen at many WLCG sites, X509 based authentication and authorization in XRootD was significantly improved to meet both security and functionality requirements. This paper presents architecture of the pull based TPC with optional X509 credential delegation and how it is implemented in native XRootD and dCache. The paper discusses technical requirements, challenges, design choices and implementation details in the WLCG storage systems, as well as in FTS/gfal2. It also outlines XRootD’s plan to support newer TPC and security models such as token based authorization.

        Speaker: Wei Yang (SLAC National Accelerator Laboratory (US))
      • 11:45
        Quality of Service (QoS) for cost-effective storage and improved performance 15m

        The anticipated increase in storage requirements for the forthcoming HL-LHC data rates is not matched by a corresponding increase in budget. This results in a short-fall in available resources if the computing models remain unchanged. Therefore, effort is being invested in looking for new and innovative ways to optimise the current infrastructure, so minimising the impact of this shortfall.

        In this paper, we describe an R&D effort targeting "Quality of Service" (QoS), as a working group within the WLCG DOMA activity. The QoS approach aims to reduce the impact of the shortfalls, and involves developing a mechanism that both allows sites to reduce the cost of their storage hardware, with a corresponding increase in storage capacity, while also supporting innovative deployments with radically reduced cost or improved performance.

        We describe the strategy this group is developing to support these innovations, along with the current status and plans for the future.

        Speaker: Mario Lassnig (CERN)
      • 12:00
        A distributed R&D storage platform implementing quality of service 15m

        Optimization of computing resources, in particular storage, the costliest one, is a tremendous challenge for the High Luminosity LHC (HL-LHC) program. Several venues are being investigated to address the storage issues foreseen for HL-LHC. Our expectation is that savings can be achieved in two primary areas: optimization of the use of various storage types and reduction of the required manpower to operate the storage.

        We will describe our work, done in the context of the WLCG DOMA project, to prototype, deploy and operate an at-scale research storage platform to better understand the opportunities and challenges for the HL-LHG era. Our multi-VO platform includes several storage technologies, from highly performant SSDs to low end disk storage and tape archives, all coordinated by the use of dCache. It is distributed over several major sites in the US (AGLT2, BNL, FNAL & MWT2) which are several tens of msec RTT apart with one extreme leg over the Atlantic in DESY to test extreme latencies. As a common definition of attributes for QoS characterizing storage systems in HEP has not yet been defined, we are using this research platform to experiment on several of them, e.g., number of copies, availability, reliability, throughput, iops and latency.

        The platform provides a unique tool to explore the technical boundaries of the ‘data-lake’ concept and its potential savings in storage and operations costs.

        We will conclude with a summary of our lessons learned and where we intend to go with our next steps.

        Speaker: Patrick Fuhrmann
      • 12:15
        The Quest to solve the HL-LHC data access puzzle. The first year of the DOMA ACCESS Working Group. 15m

        HL-LHC will confront the WLCG community with enormous data storage, management and access challenges. These are as much technical as economical. In the WLCG-DOMA Access working group, members of the experiments and site managers have explored different models for data access and storage strategies to reduce cost and complexity, taking into account the boundary conditions given by our community.

        Several of these scenarios have been studied quantitatively, such as the datalake model and incremental improvements of the current computing model with respect to resource needs, costs and operational complexity.

        To better understand these models in depth, analysis of traces of current data accesses and simulations of the impact of new concepts have been carried out. In parallel, evaluations of the required technologies took place. These were done in testbed and production environments at small and large scale.

        We will give an overview of the activities and results of the working group, describe the models and summarise the results of the technology evaluation focusing on the impact of storage consolidation in the form of datalakes, where the use of read-ahead caches (XCache) has emerged as a successful approach to reduce the impact of latency and bandwidth limitation.

        We will describe the experience and evaluation of these approaches in different environments and usage scenarios. In addition we will present the results of the analysis and modelling efforts based on data access traces of experiments.

        Speaker: Xavier Espinal (CERN)
    • 11:00 12:30
      Track 5 – Software Development: Heterogenous computing and workflow managers Riverbank R2

      Riverbank R2

      Adelaide Convention Centre

      Convener: Dr Felice Pantaleo (CERN)
      • 11:00
        GNA — high performance fitting for neutrino experiments 15m

        GNA is a high performance fitter, designed to handle large scale models with big number of parameters. Following the data flow paradigm the model in GNA is built as directed acyclic graph. Each node (transformation) of the graph represents a function, that operates on vectorized data. A library of transformations, implementing various functions, is precompiled. The graph itself is assembled at runtime in Python and may be modified without recompilation.

        High performance is achieved via several ways. The computational graph is lazily evaluated. Output data of each node is cached and recalculated only in case it is required: when one of the parameters or inputs has been changed. Transformations, subgraphs or the complete graph may be executed on GPU with data transferred lazily between CPU and GPU.

        The description of the framework as well as practical examples from Daya Bay and JUNO experiments will be presented.

        Speaker: Dr Maxim Gonchar (Joint Institute for Nuclear Research)
      • 11:15
        NUMA-aware workflow management system 15m

        Modern hardware is trending towards increasingly parallel and heterogeneous architectures. Contemporary machine processors are spread across multiple sockets, where each socket can access some system memory faster than the rest, creating non-uniform memory access (NUMA). Efficiently utilizing these NUMA machines is becoming increasingly important. This paper examines latest Intel Skylake and Xeon Phi NUMA node architectures, indicating possible performance problems for multi-threaded, data processing applications, due to the kernel thread migration (TM) mechanism, that I designed to optimize power consumption. We discuss NUMA aware CLARA workflow management system that defines proper level of vertical scaling and process affinity, associating CLARA worker threads with particular processor cores. By minimizing thread migration and context-switching cost among cores, we were able to improve the data locality and reduce the cache-coherency traffic among the cores, resulting in sizable performance improvements.

        Speaker: Vardan Gyurjyan (Jefferson Lab)
      • 11:30
        Heterogeneous data-processing optimization with CLARA’s adaptive workload orchestration 15m

        The hardware landscape used in HEP and NP is changing from homogeneous multi-core systems towards heterogeneous systems with many different computing units, each with their own characteristics. To achieve data processing maximum performance the main challenge is to place the right computing on the right hardware.
        In this paper we discuss CLAS12 charge particle tracking workload partitioning that allowed us to utilize both CPU and GPU to improve the performance. The tracking application algorithm was decomposed into micro-services that are deployed on CPU and GPU processing units, where the best features of both are intelligently combined to achieve maximum performance. In this heterogeneous environment CLARA aims to match the requirements of each micro-service to the strength of a CPU or a GPU architecture. In addition, CLARA performs load balancing to minimize idle time for both processing units. However predefined execution of a micro-service on a CPU or a GPU may not be the most optimal solution due to the streaming data-quantum size and the data-quantum transfer latency between CPU and GPU. So, we trained the CLARA workflow orchestrator to dynamically assign micro-service execution to a CPU or a GPU, based on the benchmark results analyzed for a period of the real-time data-processing.

        Speaker: Vardan Gyurjyan (Jefferson Lab)
      • 11:45
        Design Pattern for Analysis Automation on Interchangeable, Distributed Resources using Luigi Analysis Workflows 15m

        In particle physics, workflow management systems are primarily used as tailored solutions in dedicated areas such as Monte Carlo production. However, physicists performing data analyses are usually required to steer their individual workflows manually which is time-consuming and often leads to undocumented relations between particular workloads.
        We present the luigi analysis workflow (law) Python package which is based on the open-source pipelining tool luigi, originally developed by Spotify. It establishes a generic design pattern for analyses of arbitrary scale and complexity, and shifts the focus from executing to defining the analysis logic. Law provides the building blocks to seamlessly integrate with interchangeable remote resources without, however, limiting itself to a specific choice of infrastructure. In particular, it introduces the paradigm of complete separation between analysis algorithms on the one hand, and run locations, storage locations, and software environments on the other hand.
        To cope with the sophisticated demands of end-to-end HEP analyses, law supports job execution on WLCG infrastructure (ARC, gLite) as well as on local computing clusters (HTCondor, LSF), remote file access via most common protocols through the Grid File Access Library (GFAL2), and an environment sandboxing mechanism with support for Docker and Singularity containers. Moreover, the novel approach ultimately aims for analysis preservation out-of-the-box.
        Law is developed open-source and entirely experiment independent. It is successfully employed in ttH cross section measurements and searches for di-Higgs boson production with the CMS experiment.

        Speaker: Marcel Rieger (CERN)
      • 12:00
        Raythena: a vertically integrated scheduler for ATLAS applications on heterogeneous distributed resources 15m

        The ATLAS experiment has successfully integrated High-Performance Computing (HPC) resources in its production system. Unlike the current generation of HPC systems, and the LHC computing grid, the next generation of supercomputers is expected to be extremely heterogeneous in nature: different systems will have radically different architectures, and most of them will provide partitions optimized for different kinds of workloads. In this work we explore the applicability of concepts and tools realized in Ray (the high-performance distributed execution framework targeting large-scale machine learning applications) to ATLAS event throughput optimization on heterogeneous distributed resources, ranging from traditional grid clusters to Exascale computers.
        We present a prototype of Raythena, a Ray-based implementation of the ATLAS Event Service (AES), a fine-grained event processing workflow aimed at improving the efficiency of ATLAS workflows on opportunistic resources, specifically HPCs. The AES is implemented as an event processing task farm that distributes packets of events to several worker processes running on multiple nodes. Each worker in the task farm runs an event-processing application (Athena) as a daemon. In Raythena we replaced the event task farm workers with stateful components of Ray called Actors, which process packets of events and return data processing results. In addition to stateful Actors, Raythena also utilizes stateless Tasks for merging intermediate outputs produced by the Actors. The whole system is orchestrated by Ray, which assigns work to Actors and Tasks in a distributed, possibly heterogeneous, environment.
        The second thrust of this study is to use Raythena to schedule Gaudi Algorithms (the primary unit of work of ATLAS' Athena framework) across a set of heterogeneous nodes. For ease of testing, we have used the Gaudi execution flow simulator to run a production ATLAS reconstruction scenario consisting of 309 Algorithms, modeled by synthetic CPU burners constrained by the data dependencies, and run for the time duration of the original Algorithms. The Algorithms are wrapped in Ray Actors or Tasks, and communicate via the Ray Global Control Store. This approach allows the processing of a single event to be distributed across more than one node, a functionality currently not supported by the Athena framework. We will discuss Raythena features and performance as a scheduler for ATLAS workflows, comparing them to those offered by Athena.
        For all its flexibility, the AES implementation is currently comprised of multiple separate layers that communicate through ad-hoc command-line and file-based interfaces. The goal of Raythena is to integrate these layers through a feature-rich, efficient application framework. Besides increasing usability and robustness, a vertically integrated scheduler will enable us to explore advanced concepts such as dynamically shaping of workflows to exploit currently available resources, particularly on heterogeneous systems.

        Speaker: Miha Muskinja (Lawrence Berkeley National Lab. (US))
      • 12:15
        Computational workflow of the LZ dark matter detection experiment at NERSC 15m

        High Energy Physics experiments face unique challenges when running their computation on High Performance Computing (HPC) resources. The LZ dark matter detection experiment has two data centers, one each in the US and UK, to perform computations. Its US data center uses the HPC resources at NERSC.
        In this talk, I will describe the current computational workflow of the LZ experiment, detailing some of the challenges faced while making the transition from network distributed computing environments like PDSF to newer HPC resources like Cori, at NERSC.

        Speaker: Dr Venkitesh Ayyar (Lawrence Berkeley National Lab)
    • 11:00 12:30
      Track 6 – Physics Analysis: Framework Hall G

      Hall G

      Adelaide Convention Centre

      Convener: Maurizio Pierini (CERN)
      • 11:00
        Using Analysis Declarative Languages for the HL-LHC 15m

        The increase in luminosity by a factor of 100 for the HL-LHC with respect to Run 1 poses a big challenge from the data analysis point of view. It demands a comparable improvement in software and processing infrastructure. The use of GPU enhanced supercomputers will increase the amount of computer power and analysis languages will have to be adapted to integrate them. The particle physics community has traditionally developed their own tools to analyze the data, usually creating dedicated ROOT-based data formats. However, there have been several attempts to explore the inclusion of new tools into the experiments data analysis workflow considering data formats not necessarily based on ROOT. Concepts and techniques include declarative languages to specify hierarchical data selection and transformation, cluster systems to manage processing the data, Machine Learning integration at the most basic levels, statistical analysis techniques, etc. This talk will provide an overview of the current efforts in the field, including efforts in traditional programming languages like C++, Python, and Go, and efforts that have invented their own languages like Root Data Frame, CutLang, ADL, coffea, and functional declarative languages. There is a tremendous amount of activity in this field right now, and this talk will attempt to summarize the current state of the field.

        Speaker: Gordon Watts (University of Washington (US))
      • 11:15
        Evolution of the ATLAS analysis model for Run-3 and prospects for HL-LHC 15m

        With an increased dataset obtained during CERN LHC Run-2, the even larger forthcoming Run-3 data and more than an order of magnitude expected increase for HL-LHC, the ATLAS experiment is reaching the limits of the current data production model in terms of disk storage resources. The anticipated availability of an improved fast simulation will enable ATLAS to produce significantly larger Monte Carlo samples with the available CPU, which will then be limited by insufficient disk resources.
        The ATLAS Analysis Model Study Group for Run-3 was setup at the end of Run-2. Its tasks have been to analyse the efficiency and suitability of the current analysis model and to propose significant improvements to it. The group has considered options allowing ATLAS to save, for the same data/MC sample, at least 30% disk space overall, and has given directions how significantly larger savings could be realised for the HL-LHC. Furthermore, recommendations have been suggested to harmonise the current stage of analysis across the collaboration. The group has now completed its work: key recommendations will be the new small sized analysis formats DAOD_PHYS and DAOD_PHYSLITE and the increased usage of a tape carousel mode in the centralized production of these formats. This talk will review the recommended ATLAS analysis model for Run-3 and its status of the implementation. It will also provide an outlook to the HL-LHC analysis.

        Speaker: Johannes Elmsheuser (Brookhaven National Laboratory (US))
      • 11:30
        Extreme compression for Large Scale Data store 15m

        For the last 5 years Accelogic pioneered and perfected a radically new theory of numerical computing codenamed “Compressive Computing”, which has an extremely profound impact on real-world computer science. At the core of this new theory is the discovery of one of its fundamental theorems which states that, under very general conditions, the vast majority (typically between 70% and 80%) of the bits used in modern large-scale numerical computations are absolutely irrelevant for the accuracy of the end result. This theory of Compressive Computing provides mechanisms able to identify (with high intelligence and surgical accuracy) the number of bits (i.e., the precision) that can be used to represent numbers without affecting the substance of the end results, as they are computed and vary in real time. The bottom line outcome would be to provide a state-of-the-art compression algorithm that surpasses those currently available in the ROOT framework, with the purpose of enabling substantial economic and operational gains (including speedup) for High Energy and Nuclear Physics data storage/analysis. In our initial studies, a factor of nearly x4 (3.9) compression was achieved with RHIC/STAR data where ROOT compression managed only x1.4.

        In this contribution, we will present our concepts of “functionally lossless compression”, have a glance at examples and achievements in other communities, present the results and outcome of our current R&D as well as present a high-level view of our plan to move forward with a ROOT implementation that would deliver a basic solution readily integrated into HENP applications. As a collaboration of experimental scientists, private industry, and the ROOT Team, our aim is to capitalize on the substantial success delivered by the initial effort and produce a robust technology properly packaged as an open-source tool that could be used by virtually every experiment around the world as means for improving data management and accessibility.

        Speaker: Gene Van Buren (Brookhaven National Laboratory)
      • 11:45
        Data Analysis using ALICE Run3 Framework 15m

        ALICE Experiment is currently undergoing a major upgrade program, both in
        terms of hardware and software, to prepare for the LHC Run 3. A new Software
        Framework is being developed in collaboration with the FAIR experiments at GSI
        to cope with the 100 fold increase in collected collisions.
        We present our progress to adapt such a framework for the end user physics data
        analysis. In particular, we will highlight the design and technology choices.
        We will show how we adopt Apache Arrow as a platform for our in memory analysis
        data layout. We will illustrate the benefits of this solution, such as:
        efficient and parallel data processing, interoperability with a large number of
        analysis tools and ecosystems, integration with the modern ROOT declarative
        analysis framework RDataFrame.

        Speaker: Giulio Eulisse (CERN)
      • 12:00
        The Scikit-HEP project - overview and prospects 15m

        Scikit-HEP is a community-driven and community-oriented project with the goal of providing an ecosystem for particle physics data analysis in Python. Scikit-HEP is a toolset of approximately twenty packages and a few “affiliated” packages. It expands the typical Python data analysis tools for particle physicists. Each package focuses on a particular topic, and interacts with other packages in the toolset, where appropriate. Most of the packages are easy to install in many environments; much work has been done this year to provide binary “wheels” on PyPI and conda-forge packages. The uproot family provides pure Python ROOT file access and has been a runaway success with over 15000 downloads per month. AwkwardArray provides a natural “Jagged” array structure. The iMinuit package exposes the MINUIT2 C++ package to Python. Histogramming is central in any analysis workflow and has received much attention, including new Python bindings for the performant C++14 Boost::Histogram library. The Particle and DecayLanguage packages were developed to deal with particles and decay chains. Other packages provide the ability to interface between Numpy and popular HEP tools such as Pythia and FastJet. The Scikit-HEP project has been gaining interest and momentum, by building a user and developer community engaging collaboration across experiments. Some of the packages are being used by other communities, including the astroparticle physics community. An overview of the overall project and toolset will be presented, as well as a vision for development and sustainability.

        Speaker: Eduardo Rodrigues (University of Cincinnati (US))
      • 12:15
        COFFEA - Columnar Object Framework For Effective Analysis 15m

        The COFFEA Framework provides a new approach to HEP analysis, via columnar operations, that improves time-to-insight, scalability, portability, and reproducibility of analysis. It is implemented with the Python programming language and commodity big data technologies such as Apache Spark and NoSQL databases. To achieve this suite of improvements across many use cases, COFFEA takes a factorized approach, separating the analysis implementation and data delivery scheme. All analysis operations are implemented using the NumPy or awkward-array packages which are wrapped to yield user code whose purpose is quickly intuited. Various data delivery schemes are wrapped into a common front-end which accepts user inputs and code, and returns user defined outputs. We will present published results from analysis of CMS data using the COFFEA framework along with a discussion of metrics and the user experience of arriving at those results with columnar analysis.

        Speaker: Nick Smith (Fermi National Accelerator Lab. (US))
    • 11:00 12:30
      Track 7 – Facilities, Clouds and Containers: Trends and new Approaches Riverbank R7

      Riverbank R7

      Adelaide Convention Centre

      Convener: Oksana Shadura (University of Nebraska Lincoln (US))
      • 11:00
        Real-time HEP analysis with funcX, a high-performance platform for function as a service 15m

        The traditional HEP analysis model uses successive processing steps to reduce the initial dataset to a size that permits real-time analysis. This iterative approach requires significant CPU time and storage of large intermediate datasets and may take weeks or months to complete. Low-latency, query-based analysis strategies are being developed to enable real-time analysis of primary datasets by replacing conventional nested loops over objects with native operations on hierarchically nested, columnar data. Such queries are well-suited to distributed processing using a strategy called function as a service (FaaS).

        In this presentation we introduce funcX---a high-performance FaaS platform that enables intuitive, flexible, efficient, and scalable remote function execution on existing infrastructure including clouds, clusters, and supercomputers. A funcX function explicitly defines a function body and dependencies required to execute the function. FuncX allows users, interacting via a REST API, to register and then execute such functions without regard for the physical resource location or scheduler architecture on which the function is executed---an approach we refer to as ``serverless supercomputing.'' We show how funcX can be used to parallelize a real-world HEP analysis operating on columnar data to aggregate histograms of analysis products of interest in real time. Subtasks representing partial histograms are dispatched as funcX requests with expected runtimes of less than a second. Finally, we demonstrate efficient execution of such analyses on heterogeneous resources, including leadership-class computing facilities.

        Speaker: Dr Anna Elizabeth Woodard (University of Chicago)
      • 11:30
        Integrating Interactive Jupyter Notebooks at the SDCC 15m

        At the SDCC we are deploying a Jupyterhub infrastructure to enable
        scientists from multiple disciplines to access our diverse compute and
        storage resources. One major design goal was to avoid rolling out yet
        another compute backend and leverage our pre-existing resources via our
        batch systems (HTCondor and Slurm). Challenges faced include creating a
        frontend that allows users to choose what HPC resources they have access
        to as well as selecting containers or environments, delegating
        authentication to a MFA-enabled proxy, and automating deployment of
        multiple hub instances. We will show what we have done, and some
        examples of how we have worked with various groups to get their analysis
        working with Jupyter notebooks.

        Speaker: Ofer Rind
      • 11:45
        The SIMPLE Framework for deploying containerized grid services 15m

        The WLCG has over 170 sites and the number is expected to grow in the coming years. In order to support WLCG workloads, each site has to deploy and maintain several middleware packages and grid services. Setting up, maintaining and supporting the grid infrastructure at a site can be a demanding activity and often requires significant assistance from WLCG experts. Modern configuration management (Puppet, Ansible, ...), container orchestration (Docker Swarm, Kubernetes, ...) and containerization technologies (Docker, ...) can effectively make such activities lightweight via packaging sensible configurations of grid services and providing simple mechanisms to distribute and deploy them across the infrastructure available at a site. This article describes the SIMPLE project: a Solution for Installation, Management and Provisioning of Lightweight Elements. The SIMPLE framework leverages modern infrastructure management tools to deploy containerized grid services, such as popular compute elements (HTCondor, ARC, ...), batch systems (HTCondor, Slurm, ...), worker nodes etc. It is built on the principles of software sustainability, modularity and scalability. The article also describes the framework’s architecture, extensibility and the special features that enable lightweight deployments at WLCG sites.

        Speaker: Julia Andreeva (CERN)
      • 12:00
        Towards a NoOps Model for WLCG 15m

        One of the most costly factors in providing a global computing infrastructure such as the WLCG is the human effort in deployment, integration, and operation of the distributed services supporting collaborative computing, data sharing and delivery, and analysis of extreme scale datasets. Furthermore, the time required to roll out global software updates, introduce new service components, or prototype novel systems requiring coordinated deployments across multiple facilities is often increased by communication latencies, staff availability, and in many cases expertise required for operations of bespoke services. While the WLCG computing grid (and distributed systems implemented throughout HEP) is a global service platform, it lacks the capability and flexibility of a modern platform-as-a-service including continuous integration/continuous delivery (CI/CD) methods, development-operations capabilities (DevOps, where developers assume a more direct role in the actual production infrastructure), and automation. Most importantly, tooling which reduces required training, bespoke service expertise, and the operational effort throughout the infrastructure, most notably at the resource endpoints ("sites"), is entirely absent in the current model. In this paper, we explore ideas and questions around potential "NoOps" models in this context: what is realistic given organizational policies and constraints? How should operational responsibility be organized across teams and facilities? What are the technical gaps? What are the social and cybersecurity challenges? Conversely what advantages does a NoOps model deliver for innovation and for accelerating the pace of delivery of new services needed for the HL-LHC era? We will describe initial work along these lines in the context of providing a data delivery network supporting IRIS-HEP DOMA R&D.

        Speaker: Robert William Gardner Jr (University of Chicago (US))
      • 12:15
        The NOTED software tool-set enables improved exploitation of WAN bandwidth for Rucio data transfers via FTS 15m

        We describe the software tool-set being implemented in the contest of the NOTED [1] project to better exploit WAN bandwidth for Rucio and FTS data transfers, how it has been developed and the results obtained.

        The first component is a generic data-transfer broker that interfaces with Rucio and FTS. It identifies data transfers for which network reconfiguration is both possible and beneficial, translates the Rucio and FTS information into parameters that can be used by network controllers and makes these available via a public interface.

        The second component is a network controller that, based on the parameters provided by the transfer broker, decides which actions to apply to improve the path for a given transfer.

        Unlike the transfer-broker, the network controller described here is tailored to the CERN network as it has to choose the appropriate action given the network configuration and protocols used at CERN. However, this network controller can easily be used as a model for site-specific implementations elsewhere.

        The paper describes the design and the implementation of the two tools, the tests performed and the results obtained. It also analyses how the tool-set could be used for WLCG in the contest of the DOMA [2] activity.

        [1] Network Optimisation for Transport of Experimental Data - CERN project
        [2] Data Organisation, Management and Access - WLCG activity

        Speaker: Tony Cass (CERN)
    • 11:00 12:30
      Track 8 – Collaboration, Education, Training and Outreach: Outreach Riverbank R1

      Riverbank R1

      Adelaide Convention Centre

      Convener: Marzena Lapka (CERN)
      • 11:00
        The International Particle Physics Outreach Group - Reaching Across the Globe with Science 15m

        The International Particle Physics Outreach Group (IPPOG) is a network of scientists, science educators and communication specialists working across the globe in informal science education and outreach for particle physics. The primary methodology adopted by IPPOG requires the direct involvement of scientists active in current research with education and communication specialists, in order to effectively develop and share best practices in outreach. IPPOG member activities include the International Particle Physics Masterclass programme, International Day of Women and Girls in Science, Worldwide Data Day, International Muon Week and International Cosmic Day organisation, and participation in activities ranging from public talks, festivals, exhibitions, teacher training, student competitions, and open days at local institutions. These independent activities, often carried out in a variety of languages to public with a variety of backgrounds, all serve to gain the public trust and to improve worldwide understanding and support of science. We present our vision of IPPOG as a strategic pillar of particle physics, fundamental research and evidence-based decision-making around the world.

        Speaker: Steven Goldfarb (University of Melbourne (AU))
      • 11:15
        Belle2VR: An Interactive Virtual Reality Visualization of GEANT4 Event Histories 15m

        I describe a novel interactive virtual reality visualization of the Belle II detector at KEK and the animation therein of GEANT4-simulated event histories. Belle2VR runs on Oculus and Vive headsets (as well as in a web browser and on 2D computer screens, in the absence of a headset). A user with some particle-physics knowledge manipulates a gamepad or hand controller(s) to interact with and interrogate the detailed GEANT4 event history over time, to adjust the visibility and transparency of the detector subsystems, to translate freely in 3D, to zoom in or out, and to control the event-history timeline (scrub forward or backward, speed up or slow down). A non-expert uses the app - during public outreach events, for example - to explore the world of subatomic physics via electron-positron collision events in the Belle II experiment at the SuperKEKB colliding-beam facility at KEK in Japan. Multiple simultaneous users, wearing untethered locomotive VR backpacks and headsets, walk about a room containing the virtual model of the Belle II detector and each others' avatars as they observe and control the simulated event history. Developed at Virginia Tech by an interdisciplinary team of researchers in physics, education, and virtual environments, the simulation is intended to be integrated into the undergraduate physics curriculum. I describe the app, including visualization features and design decisions, and illustrate how a user interacts with its features to expose the underlying physics in each electron-positron collision event.

        Speaker: Leo Piilonen (Virginia Tech)
      • 11:30
        The CMS DAQ pinball machine 15m

        We present an interactive game for up to seven players that demonstrates the challenges of on-line event selection at the Compact Muon Solenoid (CMS) experiment to the public. The game - in the shape of a popular classic pinball machine - was conceived and prototyped by an interdisciplinary team of graphic designers, physicists and engineers at the CMS Create hackathon in 2016. Having won the competition, the prototype was turned into a fully working machine that is now exhibited on the CMS visitor's path. Teams of 2-7 visitors can compete with one another to collect as many interesting events as possible within a simulated LHC fill. In a fun and engaging way, the game conveys concepts such as multi-level triggering, pipelined processing, event building, the importance of purity in event selection and more subtle details such as dead time. The multi-player character of the game corresponds to the distributed nature of the actual trigger and data acquisition system of the experiment. We present the concept of the game, its design and its technical implementation centred around an Arduino micro-controller controlling 700 RGB LEDs and a sound subsystem running on a Mac mini.

        Speaker: Hannes Sakulin (CERN)
      • 11:45
        Engaging the youth in programming and physics through an online educational activity 15m

        The rapid economic growth is building new trends in careers. Almost every domain, including high-energy physics, needs people with strong capabilities in programming. In this evolving environment, it is highly desirable that young people are equipped with computational thinking (CT) skills, such as problem-solving and logical thinking, as well as the ability to develop software applications and write code. These are crucial elements of Science, Technology, Engineering, and Mathematics education (STEM).

        This talk will present an outcome from a Proof of Concept study of educational online activity. The project consists of building a first step of an interactive coding tutorial that will aim to introduce young people to computer science and particle physics principles in a fun and engaging way. Successful realization of this online educational asset will equip educators with a new tool to introduce STEM education and digital literacy in the classrooms, to eventually inspire young people to acquire necessary skills to be ready for a digital economic growth and future jobs.

        Speaker: Marzena Lapka (CERN)
      • 12:00
        Fluidic Data: When Art Meets CERN, Data Flows 15m

        Fluidic Data is a floor-to-ceiling installation spanning the four levels of the CERN Data Centre stairwell. It utilizes the interplay of water and light to visualize the magnitude and flow of information coming from the four major LHC experiments. The installation consists of an array of transparent hoses that house colored fluid, symbolizing the data of each experiment, surrounded by a collection of diffractive "pods" representing the particles pivotal to each experiment. The organic fusion of art and science engenders a meditative environment, allowing the visitor time for reflection and curiosity.

        The Fluidic Data installation is a cross department collaboration that incorporates materials and techniques used in the construction of the LHC and its experiments. The project brings together artists, engineers, science communicators and physicists with a common goal of communicating CERN's research and resources. The success of this collaboration exemplifies the effectiveness of working in diverse teams, both intellectually and culturally, to accomplish unique projects.

        Speaker: Julien Leduc (CERN)
      • 12:15
        Public Engagement - More than just fun 15m

        Public Engagement (PE) with science should be more than “fun” for the staff involved. PE should be a strategic aim of any publically funded science organisation to ensure the public develops an understanding and appreciation of their work, its benefits to everyday life and to ensure the next generation is enthused to take up STEM careers. Most scientific organisations do have aims to do this, but very few have significant budgets to deliver this. In a landscape of ever tightening budgets, how can we develop a sustainable culture of PE within these organisations?

        UKRI/STFC’s Scientific Computing Department present how we have worked to embed a culture of PE with the department by developing our early career staff members; highlighting the impact PE makes at the departmental and project level; and linking PE to our competency framework.

        We will also discuss how our departmental work interacts with and complements STFC’s organisational-wide PE effort, such as making use of a shared evaluation framework that allows us to evaluate our public engagement activities against their goals and make strategic decisions about the programmes future direction.

        Speaker: Mr Greg Corbett (STFC)
    • 11:00 12:30
      Track 9 – Exascale Science: Porting applications to HPCs Riverbank R4

      Riverbank R4

      Adelaide Convention Centre

      Convener: Steven Farrell (Lawrence Berkeley National Lab (US))
      • 11:00
        MPI-based tools for large-scale training and optimization at HPC sites 15m

        MPI-learn and MPI-opt are libraries to perform large-scale training and hyper-parameter optimization for deep neural networks. The two libraries, based on Message Passing Interface, allows to perform these tasks on GPU clusters, through different kinds of parallelism. The main characteristic of these libraries is their flexibility: the user has complete freedom in building her own model, thanks to the multi-backend support. In addition, the library supports several cluster architectures, allowing a deployment on multiple platforms. This generality can make this the basis for a train & optimise service for the HEP community. We present scalability results obtained from two typical HEP use-case: jet identification from raw data and shower generation from a GAN model. Results on GPU clusters were obtained at the ORNL TITAN supercomputer ad other HPC facilities, as well as exploiting commercial cloud resources and OpenStack. A comprehensive comparisons of scalability performance across platforms will be presented, together with a detailed description of the libraries and their functionalities.

        Speaker: Vladimir Loncar (University of Belgrade (RS))
      • 11:15
        Migrating Engineering Windows HPC applications to Linux HTCondor and SLURM Clusters 15m

        CERN IT department has been maintaining different HPC facilities over the past five years, one in Windows and the other one on Linux as the bulk of computing facilities at CERN are running under Linux. The Windows cluster has been dedicated to engineering simulations and analysis problems. This cluster is a High Performance Computing (HPC) cluster thanks to powerful hardware and low-latency interconnects. The Linux cluster resources are accessible through HTCondor, and are used for general purpose parallel but single-node type jobs, providing computing power to the CERN experiments and departments for tasks such as physics event reconstruction, data analysis and simulation. For HPC workloads that require multi-node parallel environments for MPI programs, there is a dedicated HPC service with MPI clusters running under the SLURM batch system and dedicated hardware with fast interconnects.

        In the past year, it was decided to consolidate compute intensive jobs in Linux to make a better use of the existing resources. Moreover, this was also in line with CERN IT strategy to reduce its dependencies on Microsoft products. This paper describes the migration of Ansys, COMSOL and CST users who were running on Windows HPC to Linux clusters. Ansys, COMSOL and CST are three engineering applications used at CERN on different domains, like multiphysics simulations or electromagnetic field problems. Users of these applications are sitting in different departments, with different needs and levels of expertise. In most cases the users have no prior knowledge of Linux. The paper will present the technical strategy to allow the engineering users to submit their simulations to the appropriate Linux cluster, depending on their HW needs. It will also describe the technical solution to integrate their Windows installations to submit to Linux clusters. Finally, the challenges and lessons learnt during the migration will be also discussed.

        Speaker: Maria Alandes Pradillo (CERN)
      • 11:30
        Geant Exascale Pilot Project 15m

        The upcoming generation of exascale HPC machines will all have most of their computing power provided by GPGPU accelerators. In order to be able to take advantage of this class of machines for HEP Monte Carlo simulations, we started to develop a Geant pilot application as a collaboration between HEP and the Exascale Computing Project. We will use this pilot to study and characterize how the machines’ architecture affects performance. The pilot will encapsulate the minimum set of physics and software framework processes necessary to describe a representative HEP simulation problem. The pilot will then be used to exercise communication, computation, and data access patterns. The project’s main objective is to identify re-engineering opportunities that will increase event throughput by improving single node performance and being able to make efficient use of the next generation of accelerators available in Exascale facilities.

        Speaker: Elizabeth Sexton-Kennedy (Fermi National Accelerator Lab. (US))
      • 11:45
        Covariance Matrix Acceleration on a Hybrid FPGA/CPU Platform 15m

        Covariance matrices are used for a wide range of applications in particle ohysics, including Kalman filter for tracking purposes, as well as for Primary Component Analysis and other dimensionality reduction techniques. The covariance matrix contains covariance and variance measures between all permutations of data dimensions, leading to high computational cost.
        By using a novel decomposition of the covariance matrix and exploiting parallelism on FPGA as well as separability of subtasks to CPU and FPGA, a linear increase of computation time for 156 number of integer dimensions and a constant computation time for 16 integer dimensions is achieved for exact covariance matrix calculation on a hybrid FPGA-CPU system, the Intel HARP 2. This leads up to 100 times faster results than the FPGA baseline and 10 times faster computation time compared to standard CPU covariance matrix calculation.

        Speaker: Lukas On Arnold (Columbia University)
      • 12:00
        3D Generative Adversarial Networks inference implementation on FPGAs 15m

        Detailed simulation is one of the most expensive tasks, in terms of time and computing resources for High Energy Physics experiments. The need for simulated events will dramatically increase for the next generation experiments, like the ones that will run at the High Luminosity LHC. The computing model must evolve and in this context, alternative fast simulation solutions are being studied. 3DGAN represent a successful example across the several R&D activities focusing on the use of deep generative models to particle detector simulation: physics results in terms of agreement to standard Monte Carlo techniques are already very promising. Optimisation of the computing resources needed to train these models, and consequently to deploy them efficiently during the inference phase will be essential to exploit the added-value of their full capabilities.

        In this context, CERN openlab has a collaboration with the researchers at SHREC at the University of Florida and with Intel to accelerate the 3DGAN inferencing stage using FPGAs. This contribution will describe the efforts ongoing at the University of Florida to develop an efficient heterogeneous computing (HGC) framework, CPUs integrated with accelerators such as GPUs and FPGAs, in order to accelerate Deep Learning. The HGC framework uses Intel distribution of OpenVINO, running on an Intel Programmable Acceleration Card (PAC) equipped with an Arria 10 GX FPGA.

        Integration of the 3DGAN use case in the HGC framework has required development and optimisation of new FPGA primitives using the Intel Deep Learning Acceleration (DLA) development suite.

        A number of details of this work and preliminary results will be presented, specifically in terms of speedup, stimulating a discussion for future development.

        Speaker: Federico Carminati (CERN)
    • 12:30 14:00
      Lunch 1h 30m
    • 14:00 15:30
      Track 1 – Online and Real-time Computing: Real-time analysis Riverbank R5

      Riverbank R5

      Adelaide Convention Centre

      Convener: Steven Schramm (Universite de Geneve (CH))
      • 14:00
        40 MHz Level-1 Trigger Scouting for CMS 15m

        The CMS experiment will be upgraded for operation at the High-Luminosity LHC to maintain and extend its optimal physics performance under extreme pileup conditions. Upgrades will include an entirely new tracking system, supplemented by a track trigger processor capable of providing tracks at Level-1, as well as a high-granularity calorimeter in the endcap region. New front-end and back-end electronics will also provide the level-1 trigger with high-resolution information from the barrel calorimeter and the muon systems. The upgraded Level-1 processors, based on powerful FPGAs, will be able to carry out sophisticated feature searches with resolutions often similar to the offline ones, while keeping pileup effects under control. In this paper, we discuss the feasibility of a system capturing Level-1 intermediate data at the beam-crossing rate of 40 MHz and carrying out online analyses based on these limited-resolution data. This 40 MHz scouting system would provide fast and virtually unlimited statistics for detector diagnostics, alternative luminosity measurements and, in some cases, calibrations, and it has the potential to enable the study of otherwise inaccessible signatures, either too common to fit in the L1 accept budget, or with requirements which are orthogonal to “mainstream” physics, such as long-lived particles. We discuss the requirements and possible architecture of a Phase-2 40 MHz scouting system, as well as some of the physics potential, and results from a demonstrator operated at the end of Run-2 using the Global Muon Trigger data from CMS. Plans for further demonstrators envisaged for Run 3 are also discussed.

        Speaker: Hannes Sakulin (CERN)
      • 14:15
        An express data production chain in the STAR experiment 15m

        Within the FAIR Phase-0 program the fast algorithms of the FLES (First-Level Event Selection) package developed for the CBM experiment (FAIR/GSI, Germany) are adapted for online and offline processing in the STAR experiment (BNL, USA). Using the same algorithms creates a bridge between online and offline. This makes it possible to combine online and offline resources for data processing.

        Thus, on the basis of the STAR HLT farm an express data production chain was created, which extends the functionality of HLT in real time, up to the analysis of physics. The same express data production chain can be used on the RCF farm, which is used for fast offline production with the similar tasks as in the extended HLT. The chain of express analysis does not interfere with the chain of standard analysis.

        An important advantage of express analysis is that it allows to start calibration, production and analysis of the data as soon as they are received. Therefore, use of the express analysis can be beneficial for BES-II data production and help accelerate science discovery by helping to obtain results within a year after the end of data acquisition.

        The specific features of express data production are given, as well as the result of online QA plots such as the real-time reconstruction of secondary decays in a BES-II environment.

        Speaker: Ivan Kisel (Johann-Wolfgang-Goethe Univ. (DE))
      • 14:30
        Trigger level analysis technique in ATLAS for Run 2 and beyond 15m

        With the unprecedented high luminosity delivered by the LHC, detector readout and data storage limitations severely limit searches for processes with high-rate backgrounds. An example of such searches is those for mediators of the interactions between the Standard Model and dark matter, decaying to hadronic jets. Traditional signatures and data taking techniques limit these searches to masses above the TeV. In order to extend the search range to lower masses on the order of 100 GeV and probe weaker couplings, the ATLAS experiment employs a range of novel trigger and analysis strategies. One of these is the trigger-level analysis (TLA), which records only trigger-level jet objects instead of the full detector information. This strategy of using only partial event information permits the use of lower jet trigger thresholds and increased recording rates with minimal impact on the total output bandwidth. We discuss the implementation of this stream and its planned updates for Run 3 and outline its technical challenges. We also present the results of an analysis using this technique, highlighting the competitiveness and complementarity with traditional data streams.

        Speaker: Antonio Boveia (Ohio State University)
      • 14:45
        Ingest pipeline for ASKAP 15m

        The Australian Square Kilometre Array Pathfinder (ASKAP) is a
        new generation 36-antenna 36-beam interferometer capable of producing
        about 2.5 Gb/s of raw data. The data are streamed from the observatory
        directly to the dedicated small cluster at the Pawsey HPC centre. The ingest
        pipeline is a distributed real time software which runs on this cluster
        and prepares the data for further (offline) processing by imaging and
        calibration pipelines. In addition to its main functionality, it turned out
        to be a valuable tool for various commissioning experiments and allowed us
        to run an interim system and achieve the first scientific results much earlier.
        I will review the architecture of the ingest pipeline, its role in the
        overall ASKAP's design as well as the lessons learned by developing a hard
        real-time application in the HPC environment.

        Speaker: Dr Maxim Voronkov (CSIRO)
      • 15:00
        Low Latency, Online Processing of the High-Bandwidth Bunch-by-Bunch Observation Data from the Transverse Damper Systems of the LHC 15m

        The transverse feedback system in LHC provides turn-by-turn, bunch-by-bunch measurements of the beam transverse position with a submicrometer resolution from 16 pickups. This results in a 16 high-bandwidth data-streams (1Gbit/s each), which are sent through a digital signal processing chain to calculate the correction kicks which are then applied to the beam. These data-streams contain valuable information about beam parameters and stability. A system that can extract and analyze these parameters and make them available for the users is extremely valuable for the accelerators physicists, machine operators, or engineers working with LHC. This paper introduces the next generation transverse observation system, which was designed specifically to allow demanding low-latency (few turns) beam parameter analysis such as passive tune extraction or transverse instability detection, while at the same time provide users around CERN with the raw data-streams in form of buffers. A new acquisition card and driver was developed that achieves a latency less than 100$\mu$s from the position being measured by the pickup to data being available for processing on the host. This data is then processed by a multitude of applications that are executed in a real-time environment that was fine-tuned for the driver and the applications. To handle the high throughput required by the analysis applications without saturating the computing resources, a combination of parallel programming techniques are used in combination with GPGPU computing.

        Speaker: Martin Soderen (CERN)
      • 15:15
        JANA2 Framework for event based and triggerless data processing 15m

        Development of the second generation JANA2 multi-threaded event processing framework is ongoing through an LDRD initiative grant at Jefferson Lab. The framework is designed to take full advantage of all cores on modern many-core compute nodes. JANA2 efficiently handles both traditional hardware triggered event data and streaming data in online triggerless environments. Development is being done in conjunction with the Electron Ion Collider development. Anticipated to be the next large scale Nuclear Physics facility constructed. The core framework is written in modern C++ but includes an integrated Python interface. The status of development and summary of the more interesting features will be presented.

        Speaker: David Lawrence (Jefferson Lab)
    • 14:00 15:30
      Track 2 – Offline Computing: Lightweight simulation and optimisation Riverbank R6

      Riverbank R6

      Adelaide Convention Centre

      Convener: Chris Pinkenburg
      • 14:00
        Gaussino - a Gaudi-based core simulation framework 15m

        The increase in luminosity foreseen in the future years of operation of the Large Hadron Collider (LHC) creates new challenges in computing efficiency for all participating experiment. To cope with these challenges and in preparation for the third running period of the LHC, the LHCb collaboration currently overhauls its software framework to better utilise modern computing architectures. This effort includes the LHCb simulation framework (Gauss).
        In this talk, we present Gaussino, an LHCb-independent simulation framework which forms the basis for LHCb's future simulation framework which incorporates the reimplemented or modernised core features of Gauss. It is built on Gaudi's functional framework making use of multiple threads. Event generation is interfaced to external generators with an example implementation of a multi-threaded Pythia8 interface being included. The detector simulation is handled by the multithreaded version of Geant4 with an interface allowing for the parallel execution of multiple events at the same time as well as for parallelism within a single event. Additionally, we present the integration of DD4hep geometry description into Gaussino to handle the detector geometry and conversion.

        Speaker: Dominik Muller (CERN)
      • 14:15
        Geant4 performance optimization in the ATLAS experiment 15m

        Software improvements in the ATLAS Geant4-based simulation are critical to keep up with the evolving hardware and increasing luminosity. Geant4 simulation currently accounts for about 50% of CPU consumption in ATLAS and it is expected to remain the leading CPU load during Run 4 (HL-LHC upgrade) with an approximately 25% share in the most optimistic computing model. The ATLAS experiment recently developed two algorithms for optimizing Geant4 performance: Neutron Russian Roulette (NRR) and range cuts for electromagnetic processes. The NRR randomly terminates a fraction of low energy neutrons in the simulation and weights energy deposits of the remaining neutrons to maintain physics performance. Low energy neutrons typically undergo many interactions with the detector material and their path becomes uncorrelated with the point of origin. Therefore, the response of neutrons can be efficiently estimated only with a subset of neutrons. Range cuts for electromagnetic processes exploit a built-in feature of Geant4 and terminate low energy electrons that originate from physics processes including conversions, the photoelectric effect, and Compton scattering. Both algorithms were tuned to maintain physics performance in ATLAS and together they bring about a 20% speedup of the ATLAS Geant4 simulation. Additional ideas for improvements currently under investigation will be also be discussed in the talk. Lastly, this talk presents how the ATLAS experiment utilizes software packages such as Intel's VTune to identify and resolve hot-spots in simulation.

        Speaker: Miha Muskinja (Lawrence Berkeley National Lab. (US))
      • 14:30
        FullSimLight: ATLAS standalone Geant4 simulation 15m

        HEP experiments simulate the detector response by accessing all needed data and services within their own software frameworks. However, decoupling the simulation process from the experiment infrastructure can be useful for a number of tasks, amongst them the debugging of new features, or the validation of multithreaded vs sequential simulation code and the optimization of algorithms for HPCs. The relevant features and data must be extracted from the framework to produce a standalone simulation application.
        As an example, the simulation of the detector response of the ATLAS experiment at the LHC is based on the Geant4 toolkit and is fully integrated in the experiment's framework "Athena". Recent developments opened the possibility of accessing a full persistent copy of the ATLAS geometry outside of the Athena framework. This is a prerequisite for running ATLAS Geant4 simulation standalone. In this talk we present the status of development of FullSimLight, a full simulation prototype that is being developed with the goal of running ATLAS standalone Geant4 simulation with the actual ATLAS geometry.
        The purpose of FullSimLight is to simplify studies of Geant4 tracking and physics processes, including on novel architectures. We will also address the challenges related to the complexity of ATLAS's geometry implementation, which precludes persistifying a complete detector description in a way that can be automatically read by standalone Geant4. This lightweight prototype is meant to ease debugging operations on the Geant4 side and to allow early testing of new Geant4 releases. It will also ease optimization studies and R&D activities related to HPC development: i.e. the possibility to offload partially/totally the simulation to GPUs/Accelerators without having to port the whole experimental infrastructure.

        Speaker: Marilena Bandieramonte (University of Pittsburgh (US))
      • 14:45
        The Heavy Photon Search Experiment Software Environment 15m

        The Heavy Photon Search (HPS) is an experiment at the Thomas Jefferson National Accelerator Facility designed to search for a hidden sector photon (A’) in fixed-target electro-production. It uses a silicon micro-strip tracking and vertexing detector inside a dipole magnet to measure charged particle trajectories and a fast lead-tungstate crystal calorimeter just downstream of the magnet to provide a trigger and to identify electromagnetic showers. The HPS experiment uses both invariant mass and secondary vertex signatures to search for the A’. The overall design of the detector follows from the kinematics of A’ production which typically results in a final state particle within a few degrees of the incoming beam. The occupancies of sensors near the beam plane are high, so high-rate detectors, a fast trigger, and excellent time tagging are required to minimize their impact and detailed simulations of backgrounds are crucial to the success of the experiment. The detector is fully simulated using the flexible and performant Geant4-based program "slic" using the xml-based "lcdd" detector description (described in previous CHEP conferences). Simulation of the readout and the event reconstruction itself are performed with the Java-based software package "hps-java." The simulation of the detector readout includes full charge deposition, drift and diffusion in the silicon wafers, followed by a detailed simulation of the readout chip and associated electronics. Full accounting of the occupancies and trigger was performed by overlaying simulated beam backgrounds. HPS has successfully completed two engineering runs and will complete its first physics run in the summer of 2019. Event reconstruction involving track, cluster and vertex finding and fitting for both simulated and real data will be described. We will begin with an overview of the physics goals of the experiment followed by a short description of the detector design. We will then describe the software tools used to design the detector layout and simulate the expected detector performance. Finally, the event reconstruction chain will be presented and preliminary comparisons of the expected and measured detector performance will be presented.

        Speaker: Norman Anthony Graf (SLAC National Accelerator Laboratory (US))
      • 15:00
        Selective background Monte Carlo simulation at Belle II 15m

        The large volume of data expected to be produced by the Belle II experiment presents the opportunity for for studies of rare, previously inaccessible processes. To investigate such rare processes in a high data volume environment necessitates a correspondingly high volume of Monte Carlo simulations to prepare analyses and gain a deep understanding of the contributing physics processes to each individual study. This resulting challenge, in terms of computing resource requirements, calls for more intelligent methods of simulation, in particular for background processes with very high rejection rates. This work presents a method of predicting in the early stages of the simulation process the likelihood of relevancy of an individual event to the target study using convolutional neural networks. The results show a robust training that is integrated natively into the existing Belle II analysis software framework, with steps taken to mitigate systematic biases induced by the early selection procedure.

        Speaker: James Kahn (Karlsruhe Institute of Technology (KIT))
      • 15:15
        Multithreaded simulation for ATLAS: challenges and validation strategy 15m

        Estimations of the CPU resources that will be needed to produce simulated data for the future runs of the ATLAS experiment at the LHC indicate a compelling need to speed-up the process to reduce the computational time required. While different fast simulation projects are ongoing (FastCaloSim, FastChain, etc.), full Geant4 based simulation will still be heavily used and is expected to consume the biggest portion of the total estimated processing time. In order to run effectively on modern architectures and profit from multi-core designs, a migration of the Athena framework to a multi-threading processing model has been performed in the last years. A multi-threaded simulation based on AthenaMT and Geant4MT enables substantial decreases in the memory footprint of jobs, largely from shared geometry and cross-sections tables. This approach scales better with respect to the multi-processing approach (AthenaMP) especially on the architectures that are foreseen to be used in the next LHC runs. In this paper we will report about the status of the multithreaded simulation in ATLAS, focusing on the different challenges of its validation process. We will demonstrate the different tools and strategies that have been used for debugging multi-threaded runs versus the corresponding sequential ones, in order to have a fully reproducible and consistent simulation result.

        Speaker: Marilena Bandieramonte (University of Pittsburgh (US))
    • 14:00 15:30
      Track 3 – Middleware and Distributed Computing: Operations & Monitoring Riverbank R3

      Riverbank R3

      Adelaide Convention Centre

      Convener: Stefan Roiser (CERN)
      • 14:00
        Operational Intelligence 15m

        In the near future, large scientific collaborations will face unprecedented computing challenges. Processing and storing exabyte datasets require a federated infrastructure of distributed computing resources. The current systems have proven to be mature and capable of meeting the experiment goals, by allowing timely delivery of scientific results. However, a substantial amount of interventions from software developers, shifters and operational teams is needed to efficiently manage such heterogeneous infrastructures. For instance, every year thousands of tickets are submitted to ATLAS and CMS issue tracking systems, hence further processed by the experiment operators. On the other hand, logging information from computing services and systems is being archived on ElasticSearch, Hadoop, and NoSQL data stores. Such a wealth of information can be exploited to increase the level of automation in computing operations by using adequate techniques, such as machine learning (ML), tailored to solve specific problems. ML models applied to the prediction of intelligent data placements and access patterns can help to increase the efficiency of resource exploitation and the overall throughput of the experiments distributed computing infrastructures. Time-series analyses may allow for the estimation of the time needed to complete certain tasks, such as processing a certain number of events or transferring a certain amount of data. Anomaly detection techniques can be employed to predict system failures, leading for example to network congestion. Recording and analyzing shifter actions can be used to automate tasks such as submitting tickets to support centers, or to suggest possible solutions to repeating issues. The Operational Intelligence project is a joint effort from various WLCG communities aimed at increasing the level of automation in computing operations. We discuss how state-of-the-art technologies can be used to build general solutions to common problems and to reduce the operational cost of the experiment computing infrastructure.

        Speaker: Alessandro Di Girolamo (CERN)
      • 14:15
        Big data solutions for CMS computing monitoring and analytics 15m

        The CMS computing infrastructure is composed by several subsystems that accomplish complex tasks such as workload and data management, transfers, submission of user and centrally managed production requests. Till recently, most subsystems were monitored through custom tools and web applications, and logging information was scattered in several sources and typically accessible only by experts. In the last year CMS computing fostered the adoption of common big data solutions based on open-source, scalable, and no-SQL tools, such as Hadoop, InfluxDB, and ElasticSearch, available through the CERN IT infrastructure. Such system allows for the easy deployment of monitoring and accounting applications using visualisation tools such as Kibana and Graphana. Alarms can be raised when anomalous conditions in the monitoring data are met, and the relevant teams are automatically notified. Data sources from different subsystems are used to build complex workflows and predictive analytics (data popularity, smart caching, transfer latency, …), and for performance studies. We describe the full software architecture and data flow, the CMS computing data sources and monitoring applications, and show how the stored data can be used to gain insights into the various subsystems by exploiting scalable solutions based on Spark.

        Speaker: Federica Legger (Universita e INFN Torino (IT))
      • 14:30
        Implementation of ATLAS Distributed Computing monitoring dashboards using InfluxDB and Grafana 15m

        For the last 10 years, the ATLAS Distributed Computing project has based its monitoring infrastructure on a set of custom designed dashboards provided by CERN-IT. This system functioned very well for LHC Runs 1 and 2, but its maintenance has progressively become more difficult and the conditions for Run 3, starting in 2021, will be even more demanding; hence a more standard code base and more automatic operations are needed. A new infrastructure has been provided by the CERN-IT Monit group, based on InfluxDB as the data store and Grafana as the display environment. ATLAS has adapted and further developed its monitoring tools to use this infrastructure for data and workflow management monitoring and accounting dashboards, expanding the range of previous possibilities with the aim of achieving a single, simpler, environment for all monitoring applications. This presentation will describe the tools used, the data flows for monitoring and accounting, the problems encountered and the solutions found.

        Speaker: Thomas Beermann (University of Innsbruck (AT))
      • 14:45
        Automatic log analysis with NLP for the CMS workflow handling 15m

        The central Monte-Carlo production of the CMS experiment utilizes the WLCG infrastructure and manages daily thousands of tasks, each up to thousands of jobs. The distributed computing system is bound to sustain a certain rate of failures of various types, which are currently handled by computing operators a posteriori. Within the context of computing operations, and operation intelligence, we propose a machine learning technique to learn from the operators with a view to reduce the operational workload and delays. This work is in continuation of CMS work on operation intelligence to try and reach accurate predictions with machine learning. We present an approach to consider the log files of the workflows as regular text to leverage modern techniques from natural language processing (NLP). In general, log files contain a substantial amount of text that is not human language. Therefore, different log parsing approaches are studied in order to map the log files' words to high dimensional vectors. These vectors are then exploited as feature space to train a model that predicts the action that the operator has to take. This approach has the advantage that the information of the log files is extracted automatically and the format of the logs can be arbitrary. In this work the performance of the log file analysis with NLP is presented and compared to previous approaches.

        Speaker: Lukas Layer (Universita e sezione INFN di Napoli (IT))
      • 15:00
        Easy-to-use data schema management scheme for RDBMS that includes the utilization of the column-store features 15m

        Relational database (RDB) and its management system (RDBMS) offer many advantages to us, such as a rich query language, maintainability gained from a concrete schema, robust and reasonable backup solutions such as differential backup, and so on. Recently, some of RDBMS has supported column-store features that offer data compression with a high level of both data size and query performance. These features are useful for data collection and management. However, it is not easy to leverage such features. First of all, RDBMS gains a reasonable performance only after a proper description of the data schema, which requires expertise.

        In this talk, we propose an easy-to-use data schema management scheme for RDBMS that includes the utilization of the column-store features. Our approach mainly focuses on time-series data. First of all, our approach supports appropriate schema generation to leverage an RDBMS that includes automatic creation of sub-tables and indexes. This is good preparation for leveraging column-store features in RDBMS.

        Along with the proposal, we implemented a prototype system on PostgreSQL-based RDBMS. Our preliminary experiments show a good performance over other ordinary approaches.

        Speaker: Tadashi Murakami (KEK)
      • 15:15
        Monitoring distributed computing beyond the traditional time-series histogram 15m

        In this work we review existing monitoring outputs and recommend some novel alternative approaches to improve the comprehension of large volumes of operations data that are produced in distributed computing. Current monitoring output is dominated by the pervasive use of time-series histograms showing the evolution of various metrics. These can quickly overwhelm or confuse the viewer due to the large number of similar looking plots. We propose a supplementary approach through the sonification of real-time data streamed directly from a variety of distributed computing services. The real-time nature of this method allows operations staff to quickly detect problems and identify that a problem is still ongoing, avoiding the case of investigating an issue a-priori when it may already have been resolved. In this paper we present details of the system architecture and provide a recipe for deployment suitable for both site and experiment teams.

        Speaker: Peter Love (Lancaster University (GB))
    • 14:00 15:30
      Track 4 – Data Organisation, Management and Access: Caching Riverbank R8

      Riverbank R8

      Adelaide Convention Centre

      Convener: Alessandra Forti (University of Manchester (GB))
      • 14:00
        Smart caching at CMS: applying AI to XCache edge services 15m

        The envisaged Storage and Compute needs for the HL-LHC will be a factor up to 10 above what can be achieved by the evolution of current technology within a flat budget. The WLCG community is studying possible technical solutions to evolve the current computing in order to cope with the requirements; one of the main focuses is resource optimization, with the ultimate objective of improving performance and efficiency as well as simplifying and reducing operation costs. As of today the storage consolidation based on a Data Lake model is considered a good candidate for addressing HL-LHC data access challenges, allowing global redundancy instead of local redundancy, dynamic adaptation of QoS, intelligent data deployment based on cost driven metrics. A Data Lake model under evaluation can be seen as a logical entity which hosts a distributed working set of analysis data. Compute power can be close to the lake, but also remote and thus completely external. In this context we expect Data caching to play a central role as a technical solution to reduce the impact of latency and reduce network load. A geographically distributed caching layer will be functional to many satellite computing centers might appear and disappear dynamically. In this talk we propose to develop a flexible and automated AI environment for smart management of the content of clustered cache systems, to optimize hardware for the service and operations for maintenance. In this talk we demonstrate a AI-based smart caching system, and discuss the implementation of training and inference facilities along with the XCache integration with the smart decision service. Finally, we evaluate the effect on smart-caches and data placement, and compare data placement algorithm with and without ML model.

        Speaker: Daniele Spiga (Universita e INFN, Perugia (IT))
      • 14:15
        CMS data access and usage studies at PIC Tier-1 and CIEMAT Tier-2 15m

        Computing needs projections for the HL-LHC era (2026+), following the current computing models, indicate that much larger resource increases would be required than those that technology evolution at a constant budget could bring. Since worldwide budget for computing is not expected to increase, many research activities have emerged to improve the performance of the LHC processing software applications, as well as to propose more efficient deployment scenarios and techniques which might alleviate the increase of expected resources for the HL-LHC. The massively increasing amounts of data to be processed leads to enormous challenges for HEP storage systems, networks and the data distribution to end-users. This is particularly important in scenarios in which the LHC data would be distributed from sufficiently small numbers of centers holding the experiment’s data. Enabling data locality via local caches on sites seems a very promising approach to hide transfer latencies while reducing the deployed storage space and number of replicas elsewhere. However, this highly depends on the workflow I/O characteristics and available network across sites. A crucial assessment is to study how the experiments are accessing and using the storage services deployed in sites in WLCG, to properly evaluate and simulate the benefits for several of the new emerging proposals within WLCG/HSF. In order to evaluate access and usage of storage, this contribution shows data access and popularity studies for the CMS Workflows executed in the Spanish Tier-1 (PIC) and Tier-2 (CIEMAT) sites supporting CMS activities, based on local and experiment monitoring data spanning more than one year. Simulations of data caches for end-user analysis data, as well as potential areas for storage savings will be reviewed.

        Speaker: Jose Flix Molina (Centro de Investigaciones Energéti cas Medioambientales y Tecno)
      • 14:30
        Moving the California distributed CMS xcache from bare metal into containers using Kubernetes 15m

        The University of California system has excellent networking between all of its campuses as well as a number of other Universities in CA, including Caltech, most of them being connected at 100 Gbps. UCSD and Caltech have thus joined their disk systems into a single logical xcache system, with worker nodes from both sites accessing data from disks at either site. This setup has been in place for a couple years now and has shown to work very well. Coherently managing nodes at multiple physical locations has however not been trivial, and we have been looking for ways to improve operations. With the Pacific Research Platform (PRP) now providing a Kubernetes resource pool spanning resources in the science DMZs of all the UC campuses, we have recently migrated the xcache services from being hosted bare-metal into containers. This talk presents our experience in both migrating to and operating in the new environment.

        Speaker: Matevz Tadel (Univ. of California San Diego (US))
      • 14:45
        Creating a content delivery network for general science on the backbone of the Internet using xcaches. 15m

        A general problem faced by computing on the grid for opportunistic users is that while delivering opportunistic cycles is simpler compared to delivering opportunistic storage. In this project we show how we integrated Xrootd caches places on the internet backbone to simulate a content delivery network for general science workflows. We will show that for some workflows on LIGO, DUNE, and general gravitational waves data reuse increase cpu efficiency while decreasing network bandwidth reuse.

        Speaker: Igor Sfiligoi (UCSD)
      • 15:00
        Implementation and performances of a DPM federated storage and integration within the ATLAS environment 15m

        With the increase of storage needs at the HL-LHC horizon, the data management and access will be very challenging for this critical service. The evaluation of possible solutions within the DOMA, DOMA-FR (IN2P3 project contribution to DOMA) and ESCAPE initiatives is a major activity to select the most optimal ones from the experiment and site point of views. The LAPP and LPSC teams have put their expertise and computing infrastructures in common to build the FR-ALPES federation and setup a DPM federated storage. Based on their experience of their Tier2 WLCG site management, their implication in the ATLAS Grid infrastructure and thanks to the flexibility of ATLAS and Rucio tools, the integration of this federation into the ATLAS grid infrastructure has been straightforward. In addition, the integrated DPM caching mechanism including volatile pools is also implemented. This infrastructure is foreseen to be a test bed for a DPM component within a DataLake. This presentation will describe the test bed (infrastructures separated by few ms in Round Trip Time unit) and its integration into the ATLAS framework. The impact on the sites and ATLAS operations of both the test bed implementation and its use will also be shown, as well as the measured performances on data access speed and reliability.

        Speaker: Stephane Jezequel (LAPP-Annecy CNRS/USMB (FR))
      • 15:15
        Analysis and modeling of data access patterns in ATLAS and CMS 15m

        Data movement between sites, replication and storage are very expensive operations, in terms of time and resources, for the LHC collaborations, and are expected to be even more so in the future. In this work we derived usage patterns based on traces and logs from the data and workflow management systems of CMS and ATLAS, and simulated the impact of different caching and data lifecycle management approaches. Data corresponding to one year of operation and covering all Grid sites have been the basis for the analysis. For selected sites, this data has been augmented by access logs from the local storage system to also include data accesses not managed via the standard experiments workflow management systems. We present the results of the studies, the tools developed and the experiences with the data analysis frameworks used, and assess the validity of both current and alternative approaches to data management from a cost perspective.

        Speaker: Markus Schulz (CERN)
    • 14:00 15:30
      Track 5 – Software Development: Software management and packaging Riverbank R2

      Riverbank R2

      Adelaide Convention Centre

      Convener: Mihaela Gheata (Institute of Space Science (RO))
      • 14:00
        Modern Software Stack Building for HEP 15m

        High-Energy Physics has evolved a rich set of software packages that need to work harmoniously to carry out the key software tasks needed by experiments. The problem of consistently building and deploying these software packages as a coherent software stack is one that is shared across the HEP community. To that end the HEP Software Foundation Packaging Working Group has worked to identify common solutions that can be used across experiments, with an emphasis on consistent, reproducible builds and easy deployment into CVMFS or containers via CI systems. We based our approach on well identified use cases and requirements from many experiments. In this paper we summarise the work of the group in the last year and how we have explored various approaches based on package managers from industry and the scientific computing community.

        We give details about a solution based on the Spack package manager which has been used to build the software required by the SuperNEMO and FCC experiments. We shall discuss changes that needed to be made to Spack to satisfy all our requirements. A layered approach to packaging with Spack, that allows build artefacts to be shared between different experiments, is described. We show how support for a build environment for software developers is provided.

        Speaker: Graeme A Stewart (CERN)
      • 14:15
        Gentoo Prefix as a physics software manager 15m

        In big physics experiments, as simulation, reconstruction and analysis become more sophisticated, scientific reproducibility is not a trivial task. Software is one of the biggest challenges. Modularity is a common sense of software engineering to facilitate quality and reusability of code. However, that often introduces nested dependencies not obvious for physicists to work with. Package manager is the widely practised solution to organize dependencies systematically.

        Portage from Gentoo Linux is both robust and flexible, and is highly regarded by the free operating system community. In form of Gentoo Prefix, portage can be deployed by a normal user into a directory prefix, on a workstation, cloud or supercomputing node. Software is described by its build recipes along with dependency relations. Real world use cases of Gentoo Prefix in neutrino and dark matter experiments will be demonstrated, to show how physicists could benefit from existing tools of proven superiority to guarantee reproducibility in simulation, reconstruction and analysis of big physics experiments.

        Speaker: Prof. Benda Xu (Tsinghua University)
      • 14:30
        SpackDev: Parallel Package Development with Spack 15m

        Development of scientific software has always presented challenges to its practitioners, among other things due to its inherently collaborative nature. Software systems often consistent of up to several dozen closely-related packages developed within a particular experiment or related ecosystem, with up to a couple of hundred externally-sourced dependencies. Making improvements to one such package can require related changes to multiple other packages, and some systemic improvements can require major structural changes across the ecosystem.

        There have been several attempts to produce a multi-package development system within HEP in the past, such systems usually being limited to one or a few experiments and requiring a common build system (e.g. Make, CMake). Common features include a central installation of each "release" of the software system to avoid multiple builds of the same package on a system, and integration with version control systems.

        SpackDev is based on the powerful Spack build and packaging system in wide use in HPC, utilizing its package recipes and build mangement system to extract build instructions and manage the parallel development, build and test process for multiple packages at a time. Intended to handle packages without restriction to one internal build system, SpackDev is integrated with Spack as a command extension and is generally applicable outside HEP. We describe SpackDev's features and development over the last two years and the medium-term future, and initial experience using the SpackDev in the context of the LArSoft liquid argon detector toolkit.

        Speaker: Elizabeth Sexton-Kennedy (Fermi National Accelerator Lab. (US))
      • 14:45
        Sustainable software packaging for end users with conda 15m

        The conda package manager is widely used in both commercial and academic high-performance computing across a wide range of fields. In 2016 conda-forge was founded as a community-driven package repository which allows packaging efforts to be shared across communities. This is especially important with the challenges faced when packaging modern software with complex dependency chains or specialised hardware such as GPUs. Conda-forge receives support from Anaconda Inc. and became an officially supported PyData project in 2018. Conda is a language independent package manager which can be used for providing native binaries for Linux, macOS and Windows with x86, arm64 and POWER architectures.

        The ROOT framework is a fundamental component of many HEP experiments. However, quickly installing ROOT on a new laptop or deploying it in continuous integration systems typically requires a non-negligible amount of domain-specific skills. The ability to install ROOT within conda has been requested for many years and its appeal was proven with it over 18,000 downloads within the first 5 months of it being made available. In addition, it has subsequently been used as a base for distributing other packages such as CMS’s event display package (Fireworks) and the alphatwirl analysis framework.

        In this contribution we will discuss the process of adding ROOT releases to conda-forge and how nightly builds of ROOT are being provided to allow end users to provide feedback on new and experimental features such as RDataFrame. We also discuss our experience distributing conda environments using CVMFS for physics analysts to use both interactively and with distributed computing resources.

        Speaker: Chris Burr (CERN)
      • 15:00
        CERN AppStore: Development of a multi-platform application management system for BYOD devices at CERN 15m

        The number of BYOD continuously grows at CERN. Additionally, it is desirable to move from a centrally managed model to a distributed model where users are responsible for their own devices. Following this strategy, the new tools have to be provided to distribute and - in case of licensed software - also track applications used by CERN users. The available open source and commercial solutions were analyzed and none of them proved to be a good fit for CERN use cases. Therefore, it was decided to develop a system that could integrate various open source solutions and provide desired functionality for multiple platforms, both mobile and desktop. This paper presents the architecture and design decisions made to achieve a platform-independent, modern, maintainable and extensible system for software distribution at CERN.

        Speaker: Tamas Bato (CERN)
      • 15:15
        Chopin Management System: improving Windows infrastructure monitoring and management 15m

        CERN Windows server infrastructure consists of about 900 servers. The management and maintenance is often a challenging task as the data to be monitored is disparate and has to be collected from various sources. Currently, alarms are collected from the Microsoft System Center Operation Manager (SCOM) and many administrative actions are triggered through e-mails sent by various systems or scripts.

        The objective of the Chopin Management System project is to maximize automation and facilitate the management of the infrastructure. The current status of the infrastructure, including essential health checks, is centralized and presented through a dashboard. The system collects information necessary for managing the infrastructure in the real-time, such as hardware configuration or Windows updates, and reacts to any change or failure instantly . As part of the system design, big data streaming technologies are employed in order to assure the scalability and fault-tolerance of the service, should the number of servers drastically grow. Server events are aggregated and processed in real-time through the use of these technologies, ensuring quick response to possible failures. This paper presents details of the architecture and design decisions taken in order to achieve a modern, maintainable and extensible system for Windows Server Infrastructure management at CERN.

        Speaker: Sebastian Bukowiec (CERN)
    • 14:00 15:30
      Track 6 – Physics Analysis: Lattice QCD Hall G

      Hall G

      Adelaide Convention Centre

      Convener: Phiala Shanahan (Massachusetts Institute of Technology)
      • 14:00
        GUM: GAMBIT Universal Models 15m

        GUM is a new feature of the GAMBIT global fitting software framework, which provides a direct interface between Lagrangian level tools and GAMBIT. GUM automatically writes GAMBIT routines to compute observables and likelihoods for physics beyond the Standard Model. I will describe the structure of GUM, the tools (within GAMBIT) it is able to create interfaces to, and the observables it is able to compute.

        Speaker: Sanjay Bloor (Imperial College London)
      • 14:15
        Computing the properties of nuclei from QCD 15m

        I will discuss recent advances in lattice QCD from the physics and computational points of view that have enabled basic a number properties and interactions of light nuclei to be determined directly from QCD. These calculations offer the prospect of providing nuclear matrix inputs necessary for a range of intensity frontier experiments (DUNE, mu2e) and dark matter direct-detection experiments along with well-quantified uncertainties.

        Speaker: Dr William Detmold (MIT)
      • 14:30
        Computing the magnetic field response of the proton 15m

        Background field methods offer an approach through which fundamental non-perturbative hadronic properties can be studied. Lattice QCD is the only ab initio method with which Quantum Chromodynamics can be studied at low energies; it involves numerically calculating expectation values in the path integral formalism. This requires substantial investment in high performance super computing resources. Here the background field method is used with lattice QCD to induce a uniform background magnetic field.
        A particular challenge of lattice QCD is isolating the desired state, rather than a superposition of excited states. While extensive work has been performed which allows the ground state to be identified in lattice QCD calculations, this remains a challenging proposition for the ground state in the presence of a background field. Quark level operators are introduced to resolve this challenge and thus allow for extraction of the magnetic polarisability that characterises the response of the nucleon to a magnetic field.

        Speaker: Ryan Bignell (University of Adelaide)
      • 14:45
        Investigating the anomalous magnetic moment on the lattice 15m

        There exists a long standing discrepancy of around 3.5 sigma between experimental measurements and standard model calculations of the magnetic moment of the muon. Current experiments aim to reduce the experimental uncertainty by a factor of 4, and Standard Model calculations must also be improved by a similar order. The largest uncertainty in the Standard Model calculation comes from the QCD contribution, in particular the leading order hadronic vacuum polarisation (HVP). To calculate the HVP contribution, we use lattice gauge theories which allows us to study QCD at low energies. In order to better understand this quantity, we investigate the effect of QED corrections to the leading order HVP term by including QED in our lattice calculations, and investigate flavour breaking effects. This is done using fully dynamical QCD+QED gauge configurations generated by the QCDSF collaboration and a novel method of quark turning.

        Speaker: Alex Westin (The University of Adelaide)
      • 15:00
        Directly calculating the glue component of the nucleon in lattice QCD 15m

        Computing the gluon component of momentum in the nucleon is a difficult and computationally expensive problem, as the matrix element involves a quark-line-disconnected gluon operator which suffers from ultra-violet fluctuations. But also necessary for a successful determination is the non-perturbative renormalisation of this operator. We investigate this renormalisation here by direct computation in the RI mom scheme. A clear statistical signal is obtained in the direct calculation by an adaption of the Feynman-Hellmann technique. A comparison is conducted in order to verify the energy-momentum sum rule of the nucleon.

        Speaker: Tomas Howson (University of Adelaide)
      • 15:15
        The computational challenge of lattice chiral symmetry - Is it worth the expense? 15m

        The origin of the low-lying nature of the Roper resonance has been the subject of significant interest for many years, including several investigations using lattice QCD. It has been claimed that chiral symmetry plays an important role in our understanding of this resonance. We present results from our systematic examination of the potential role of chiral symmetry in the low-lying nucleon spectrum through the direct comparison of the clover and overlap fermion actions. After a brief summary of the background motivation, we specify the computational details of the study and outline our comparison methodologies. We do not find any strong evidence supporting the claim that chiral symmetry plays a significant role in understanding the Roper resonance on the lattice.

        Speaker: Adam Virgili
    • 14:00 15:30
      Track 7 – Facilities, Clouds and Containers: Infrastructure Riverbank R7

      Riverbank R7

      Adelaide Convention Centre

      Convener: Sang Un Ahn (Korea Institute of Science & Technology Information (KR))
      • 14:00
        WLCG Web Proxy Auto Discovery for Dynamically Created Web Proxies 15m

        The WLCG Web Proxy Auto Discovery (WPAD) service provides a convenient mechanism for jobs running anywhere on the WLCG to dynamically discover web proxy cache servers that are nearby. The web proxy caches are general purpose for a number of different http applications, but different applications have different usage characteristics and not all proxy caches are engineered to work with the heaviest loads. For this reason, the initial sources of information for WLCG WPAD were the static configurations that ATLAS and CMS maintain for the Conditions data that they read through the Frontier Distributed Database system, which is the most demanding popular WLCG application for web proxy caches. That works well for use at traditional statically defined WLCG sites, but now that usage of commercial clouds is increasing, there is also a need for web proxy caches to dynamically register themselves as they are created. A package called Shoal had already been created to manage dynamically created web proxy caches. This paper describes the integration of the Shoal package into the WLCG WPAD system, such that both staticly and dynamically created web proxy caches can be located from a single source. It also describes other improvements to the WLCG WPAD system since the last CHEP publication.

        Speaker: Oliver Gutsche (Fermi National Accelerator Lab. (US))
      • 14:15
        Designing a new infrastructure for ATLAS Online Web Services 15m

        Within the ATLAS detector, the Trigger and Data Acquisition system is responsible for the online processing of data streamed from the detector during collisions at the Large Hadron Collider (LHC) at CERN. The online farm is composed of ~4000 servers processing the data read out from ~100 million detector channels through multiple trigger levels. The capability to monitor the ongoing data taking and all the involved applications is essential to debug and intervene promptly to ensure efficient data taking. The base of the current web service architecture was designed a few years ago, at the beginning of the ATLAS operation (Run 1). It was intended to serve primarily static content from a Network-attached Storage, and privileging strict security, using separate web servers for internal (ATLAS Technical and Control Network - ATCN) and external (CERN General Purpose Network and public internet) access. During these years, it has become necessary to add to the static content an increasing number of dynamic web-based User Interfaces, as they provided new functionalities and replaced legacy desktop UIs. These are typically served by applications on VMs inside ATCN and made accessible externally via chained reverse HTTP proxies. As the trend towards Web UIs continues, the current design has shown its limits, and its increasing complexity became an issue for maintenance and growth. It is, therefore, necessary to review the overall web services architecture for ATLAS, taking into account the current and future needs of the upcoming LHC Run 3.
        In this paper, we present our investigation and roadmap to re-design the web services system to better operate and monitor the ATLAS detector, while maintaining the security of critical services, such as Detector Control System, and maintaining the separation of remote monitoring and on-site control according to ATLAS policies.

        Speaker: Diana Scannicchio (University of California Irvine (US))
      • 14:30
        Construction of a New Data Center at BNL 15m

        Computational science, data management and analysis have been key factors in the success of Brookhaven Lab's scientific programs at the Relativistic Heavy Ion Collider (RHIC), the National Synchrotron Light Source (NSLS-II), the Center for Functional Nanomaterials (CFN), and in biological, atmospheric, and energy systems science, Lattice Quantum Chromodynamics (LQCD) and Materials Science as well as our participation in international research collaborations, such as the ATLAS Experiment at Europe's Large Hadron Collider (LHC) and Belle II Experiment at KEK (Japan). The construction of a new data center is an acknowledgement of the increasing demand for computing and storage services at BNL.

        The Computing Facility Revitalization (CFR) project is aimed at repurposing the former National Synchrotron Light Source (NSLS-I) building as the new datacenter for BNL. The new data center is to become available in early 2021 for ATLAS compute, disk storage and tape storage equipment, and later that year - for all other collaborations supported by the RACF/SDCC Facility, including: STAR, PHENIX and sPHENIX experiments at RHIC collider at BNL, Belle II Experiment at KEK (Japan), and BNL CSI HPC clusters. Migration of the majority of IT payload from the existing datacenter to the new datacenter is expected to begin with the central networking systems and first BNL ATLAS Tier-1 Site tape robot in early FY21, and it is expected to continue throughout FY21-23. This presentation will highlight the key MEP facility infrastructure components of the new data center. Also, we will describe our plans to migrate IT equipment between datacenters, the inter-operational period in FY21, gradual IT equipment replacement in FY21-24, and show the expected state of occupancy and infrastructure utilization for both datacenters in FY25.

        Speaker: Shigeki Misawa (BNL)
      • 14:45
        Preparing CERN Tier-0 data centres for LHC Run3 15m

        Since 2013 CERN’s local data centre combined with a colocation infrastructure at the Wigner data centre in Budapest have been hosting the compute and storage capacity for WLCG Tier-0. In this paper we will describe how we try to optimize and improve the operation of our local data centre to meet the anticipated increment of the physics compute and storage requirements for Run3, taking into account two important changes on the way: the end of the colocation contract with Wigner in 2019 and the loan of 2 out of 6 prefabricated compute containers being commissioned by the LHCb experiment for their online computing farm.

        Speaker: Olof Barring (CERN)
      • 15:00
        Computing Activities at the Spanish Tier-1 and Tier-2s for the ATLAS experiment towards the LHC Run3 and High Luminosity (HL-LHC) periods 15m

        The ATLAS Spanish Tier-1 and Tier-2s have more than 15 years of experience in the deployment and development of LHC computing components and their successful operations. The sites are already actively participating in, and even coordinating, emerging R&D computing activities developing the new computing models needed in the LHC Run3 and HL-LHC periods.
        In this contribution, we present details on the integration of new components, such as HPC computing resources, to execute ATLAS simulation workflows; the development of new techniques to improve efficiency in a cost-effective way, such as storage and CPU federations; and improvements in Data Organization, Management and Access through storage consolidations ("data-lakes"), the use of data Caches, and improving experiment data catalogues, like Event Index. The design and deployment of novel analysis facilities using GPUs together with CPUs and techniques like Machine Learning will also be presented.
        ATLAS Tier-1 and Tier-2 sites in Spain are, and will be, contributing to significant R&D in computing, evaluating different models for improving performance of computing and data storage capacity in the LHC High Luminosity era.

        Speaker: Santiago Gonzalez De La Hoz (Univ. of Valencia and CSIC (ES))
      • 15:15
        Beyond HEP: Photon and accelerator science computing infrastructure at DESY 15m

        DESY is one of the largest accelerator laboratories in Europe, developing and operating state of the art accelerators, used to perform fundamental science in the areas of high-energy physics photon science and accelerator development.\newline

        While for decades high energy physics has been the most prominent user of the DESY compute, storage and network infrastructure, various scientific areas as science with photons and accelerator development have catched up and are now dominating the demands on the DESY infrastructure resources, with significant consequences for the IT resource provisioning. In this contribution, we will present an overview of the computational, storage and network resources covering the various physics communities on site.\newline

        Ranging from HTC batch-like offline processing in the Grid and the interactive user analyses resources in the National Analysis Factory for
        the HEP community, to the computing needs of accelerator development or of photon sciences such as PETRA III or the European XFEL. Since DESY co-hosts these experiments and their data taking, their requirements include fast low-latency online processing for data taking and calibration as well as offline processing, thus HPC workloads, that are run on the dedicated {\it Maxwell} HPC cluster.\newline

        As all communities face in the coming years significant challenges due to changing environments and increasing data rates, we will discuss how this will reflect in necessary changes to the computing and storage
        infrastructures.\newline
        We will present DESY compute cloud and container orchestration plans as a possible basis for infrastructure and platform services. We will show examples of Jupyter for small scale interactive analysis, as well as its integration into large scale resources such as batch systems or Spark clusters.\newline

        To overcome the fragmentation of the various resources for all scientific communities at DESY , we explore how to integrate them into a seamless user experience in an {\it Interdisciplinary Data and Analysis Facility}

        Speaker: Thomas Hartmann (Deutsches Elektronen-Synchrotron (DE))
    • 14:00 15:30
      Track 8 – Collaboration, Education, Training and Outreach: Open data Riverbank R1

      Riverbank R1

      Adelaide Convention Centre

      Convener: Tibor Simko (CERN)
      • 14:00
        Open data provenance and reproducibility: a case study from publishing CMS open data 15m

        In this paper we present the latest CMS open data release published on the CERN Open Data portal. The samples of raw datasets, collision and simulated datasets were released together with the detailed information about the data provenance. The data production chain covers the necessary compute environments, the configuration files and the computational procedures used in each data production step. We describe data curation techniques used to obtain and publish the data provenance information and we study the possibility to reproduce parts of the released data using the publicly available information. The present work demonstrates the usefulness of releasing selected samples of raw and primary data in order to fully ensure the completeness of information about data production chain for the attention of general data scientists and other non-specialists interested in using particle physics data for education or research purposes.

        Speaker: Tibor Simko (CERN)
      • 14:15
        Using CMS Open Data for education, outreach and software benchmarking 15m

        The CMS collaboration at the CERN LHC has made more than one petabyte of open data available to the public, including large parts of the data which formed the basis for the discovery of the Higgs boson in 2012. Apart from their scientific value, these data can be used not only for education and outreach, but also for open benchmarks of analysis software. However, in their original format, the data cannot be accessed easily without experiment-specific knowledge and skills. Work is presented that allows to set up open analyses that are performed close to the published ones, but which meet minimum requirements for experiment-specific knowledge and software. The suitability of this approach for education and outreach is demonstrated with analyses that have been made fully accessible to the public via the CERN open data portal. In the second part of the talk, the value of these data as basis for benchmarks of analysis software under realistic conditions of a high-energy physics experiment is discussed.

        Speaker: Stefan Wunsch (KIT - Karlsruhe Institute of Technology (DE))
      • 14:30
        Open Data Science Mesh: friction-free collaboration for researchers bridging High-Energy Physics and European Open Science Cloud 15m

        Open Data Science Mesh (CS3MESH4EOSC) is a newly funded project to create a new generation, interoperable federation of data and higher-level services to enable friction-free collaboration between European researchers.

        This new EU-funded project brings together 12 partners from the CS3 community (Cloud Synchronization and Sharing Services). The consortium partners include CERN, Danish Technical University (DK), SURFSARA (NL), Poznan Supercomputing Centre (PL), CESNET (CZ), AARNET (AUS), SWITCH (CH), University of Munster (DE), Ailleron SA (PL), Cubbit (IT), Joint Research Centre (BE) and Fundacion ESADE (ES). CERN acts as project coordinator.

        The consortium already operates services and storage-centric infrastructure for around 300 thousand scientists and researchers across the globe. The project will integrate these local existing sites and services into a seamless mesh infrastructure which is fully interconnected with the EOSC-Hub, as proposed in the European Commission’s Implementation Roadmap for EOSC.

        The project will provide a framework for applications in several major areas: Data Science Environments, Open Data Systems, Collaborative Documents, On-demand Large Dataset Transfers and Cross-domain Data Sharing.

        The collaboration between the users will be enabled by a simple sharing mechanism: a user will select a file or folder to share with other users at other sites. Such shared links will be established and removed dynamically by the users from a streamline web interface of their local storage systems. The mesh will automatically and contextually enable different research workflow actions based on type of content shared in the folder. One of the excellence areas of CS3 services is access to content from all types of devices: web, desktop applications and mobile devices. The project augments this capability to access content stored on remote sites and will in practice introduce FAIR principles in European Science.

        The project with leverage on technologies developed and integrated in the research community, such as ScienceBox (CERNBox, SWAN, EOS), EGI-CheckIn, File Transfer Service (FTS), ARGO, EduGAIN and others. The project will also involve commercial cloud providers, integrating their software and services

        Speaker: Jakub Moscicki (CERN)
      • 14:45
        ATLAS Open Data software: the development of simple-but-real HEP data analysis examples 15m

        The ATLAS Collaboration is releasing a new set of recorded and simulated data samples at a centre-of-mass energy of 13 TeV. This new dataset was designed after an in-depth review of the usage of the previous release of samples at 8 TeV. That review showed that capacity-building is one of the most important and abundant uses of public ATLAS samples. To fulfil the requirements of the community and at the same time attract new users and use cases, we developed real analysis software based on ROOT in two of the most popular programming languages: C++ and Python. These so-called analysis frameworks are complex enough to reproduce with reasonable accuracy the results -figures and final yields- of published ATLAS Collaboration physics papers, but still light enough to be run on commodity computers. Computers that university students and regular classrooms have, allow students to explore LHC data with similar techniques to those used by current ATLAS analysers. We present the development path and the final result of these analysis frameworks, their products and how they are distributed to final users inside and outside the ATLAS community.

        Speaker: Farid Ould-Saada (University of Oslo (NO))
      • 15:00
        ATLAS Open Data: Using open education resources effectively 15m

        Perform data analysis and visualisation on your own computer? Yes, you can! Commodity computers are now very powerful in comparison to only a few years ago. On top of that, the performance of today's software and data development techniques facilitates complex computation with fewer resources. Cloud computing is not always the solution, and reliability or even privacy is regularly a concern. While the Infrastructure as a Service (IaaS) and Software as a Service (SaaS) philosophies are a key part of current scientific endeavours, there is a misleading feeling that we need to have remote computers to do any kind of data analysis. One of the aims of the ATLAS Open Data project is to provide resources — data, software and documents — that can be stored and executed in computers with minimal or non-internet access, and in as many different operating systems as possible. This approach is viewed as complementary to the IaaS/SaaS approach, where local university, students and trainers' resources can be used in an effective and reproducible way — making the HEP and Computer Sciences fields accessible to more people. We present the latest developments in the production and use of local Virtual Machines and Docker Containers for the development of physics data analysis. We also discuss example software and Jupyter notebooks, which are in constant development for use in classrooms, and students’ and teachers' computers around the world.

        Speaker: Leonid Serkin (INFN Gruppo Collegato di Udine and ICTP Trieste (IT))
      • 15:15
        Dataset of tau neutrino interactions recorded by OPERA experiment 15m

        We describe the dataset of very rare events recorded by the OPERA experiment. Those events represent tracks of particles associated with tau neutrinos emerged from a pure muon neutrino beam, due to neutrino oscillations. The OPERA detector, located in the underground Gran Sasso Laboratory, consisted of an emulsion/lead target with an average mass of about 1.2 kt, complemented by the electronic detectors. It was exposed, from 2008 to 2012, to the CNGS (CERN Neutrinos to Gran Sasso) beam, an almost pure muon neutrino beam with a baseline of 730 km, collecting a total of $17.97 \times 10^{19}$ protons on target. The OPERA Collaboration eventually assessed the discovery of $\nu_{\mu} \rightarrow \nu_{\tau}$ oscillations with a significance of 6.1 $\sigma$ by observing ten $\nu_{\tau}$ candidates. These events have been published at CERN Open Data Portal.

        Speaker: Tibor Simko (CERN)
    • 14:00 15:30
      Track X – Crossover sessions: Optimisation and acceleration Riverbank R4

      Riverbank R4

      Adelaide Convention Centre

      Conveners: Teng Jian Khoo (Universite de Geneve (CH)), Yu Nakahama Higuchi (Nagoya University (JP))
      • 14:00
        FPGA-accelerated machine learning inference as a service for particle physics computing 15m

        Large-scale particle physics experiments face challenging demands for high-throughput computing resources both now and in the future. New heterogeneous computing paradigms on dedicated hardware with increased parallelization, such as Field Programmable Gate Arrays (FPGAs), offer exciting solutions with large potential gains. The growing applications of machine learning algorithms in particle physics for simulation, reconstruction, and analysis are naturally deployed on such platforms. We demonstrate that the acceleration of machine learning inference as a web service represents a heterogeneous computing solution for particle physics experiments that requires minimal modification to the current computing model. As examples, we retrain the ResNet50 convolutional neural network to demonstrate state-of-the-art performance for top quark jet tagging at the LHC and apply a ResNet50 model with transfer learning for neutrino event classification. Using Microsoft Azure Machine Learning deploying Intel FPGAs to accelerate the ResNet50 image classification model, we achieve average inference times of 60 (10) milliseconds with our experimental physics software framework deployed as a cloud (edge or on-premises) service, representing an improvement by a factor of approximately 30 (175) in model inference latency over traditional CPU inference in current experimental hardware. A single FPGA service accessed by many CPUs achieves a throughput of 600-700 inferences per second using an image batch of one, comparable to large batch-size GPU throughput and significantly better than small batch-size GPU throughput. Deployed as an edge or cloud service for the particle physics computing model, coprocessor accelerators can have a higher duty cycle and are potentially much more cost-effective.

        Speaker: Kevin Pedro (Fermi National Accelerator Lab. (US))
      • 14:15
        GPU-based reconstruction and data compression at ALICE during LHC Run 3 15m

        In LHC Run 3, ALICE will increase the data taking rate significantly to 50 kHz continuous read out of minimum bias Pb-Pb collisions. The reconstruction strategy of the online offline computing upgrade foresees a first synchronous online reconstruction stage during data taking enabling detector calibration, and a posterior calibrated asynchronous reconstruction stage. The significant increase in the data rate poses challenges for online and offline reconstruction as well as for data compression. Compared to Run 2, the online farm must process 50 times more events per second and achieve a higher data compression factor. ALICE will rely on GPUs to perform real time processing and data compression of the Time Projection Chamber (TPC) detector in real time, the biggest contributor to the data rate. With GPUs available in the online farm, we are evaluating their usage also for the full tracking chain during the asynchronous reconstruction for the silicon Inner Tracking System (ITS) and Transition Radiation Detector (TRD). The software is written in a generic way, such that it can also run on processors on the WLCG with the same reconstruction output. We give an overview of the status and the current performance of the reconstruction and the data compression implementations on the GPU for the TPC and for the global reconstruction.

        Speaker: David Rohr (CERN)
      • 14:30
        Reconstruction of track candidates at the LHC crossing rate using FPGAs 15m

        In 2021 the LHCb experiment will be upgraded, and the DAQ system will be based on full reconstruction of events, at the full LHC crossing rate. This requires an entirely new system, capable of reading out, building and reconstructing events at an average rate of 30 MHz. In facing this challenge, the system could take advantage of a fast pre-processing of data on dedicated FPGAs. We present the results of an R&D on these technologies developed in the context of the LHCb Upgrade I. In particular, we discuss the details and potential benefits of an approach based on producing in real-time sorted collections of hits in the VELO detector (pre-tracks). These pre-processed data can then be used as seeds by the High Level Trigger (HLT) farm to find tracks for the Level 1 trigger with much lower computational effort than possible by starting from the raw detector data, thus freeing an important fraction of the power of the CPU farm for higher level processing tasks.

        Speaker: Giulia Tuci (Universita & INFN Pisa (IT))
      • 14:45
        Quantum annealing algorithms for track pattern recognition 15m

        The pattern recognition of the trajectories of charged particles is at the core of the computing challenge for the HL-LHC, which is currently the center of a very active area of research. There has also been rapid progress in the development of quantum computers, including the D-Wave quantum annealer. In this talk we will discuss results from our project investigating the use of annealing algorithms for pattern recognition. We will present results we achieved expressing pattern recognition as a Quadratic Unconstrained Binary Optimization (QUBO) that can be solved using a D-Wave Quantum Annealer. We generated QUBOs that encode the pattern recognition problem at the LHC on the TrackML dataset, and we solved them using D-Wave qbsolv hybrid optimizer. These achieved a performance exceeding 99% for purity, efficiency, and for the TrackML score at low track multiplicities. We will discuss how the algorithm performs at track multiplicities expected at the HL-LHC. We will also report on early results comparing digital annealers to quantum annealers. We will also discuss results from the application of annealing algorithms to resolve between tracks in the dense cores of jets, and possible improvement of the annealing algorithm in a new workflow with a quantum/classical hybrid optimizer. We will conclude with future perspectives on using annealing-based algorithms for pattern recognition in high-energy physics experiments.

        Speaker: Masahiko Saito (University of Tokyo (JP))
      • 15:00
        The Tracking Machine Learning Challenge 15m

        The HL-LHC will see ATLAS and CMS see proton bunch collisions reaching track multiplicity up to 10.000 charged tracks per event. Algorithms need to be developed to harness the increased combinatorial complexity. To engage the Computer Science community to contribute new ideas, we have organized a Tracking Machine Learning challenge (TrackML). Participants are provided events with 100k 3D points, and are asked to group the points into tracks; they are also given a 100GB training dataset including the ground truth. The challenge is run in two phases. The first "Accuracy" phase has run on Kaggle platform from May to August 2018; algorithms were judged judged only on a score related to the fraction of correctly assigned hits. The second "Throughput" phase ran Sep 2018 to March 2019 on Codalab, required code submission; algorithms were then ranked by combining accuracy and speed. The first phase has seen 653 participants, with top performers with innovative approaches (see arXiv:1904.06778). The second phase has recently finished and featured some astonishingly fast solutions. A "grand Finale" workshop will have taken place at CERN early July 2019. The talk will report on the lessons from the TrackML challenge and perspectives

        Speaker: Jean-Roch Vlimant (California Institute of Technology (US))
      • 15:15
        Evolving Geant4 to cope with the new HEP computing challenges 15m

        The future High Energy Physics experiments, based on upgraded or next generation particle accelerators with higher luminosity and energy, will put more stringent demands on the simulation as far as precision and speed are concerned. In particular, matching the statistical uncertainties of the collected experimental data, will require the simulation toolkits to be more CPU-efficient, while keeping the same, if not higher, precision of the physics. On the other hand, the computing architectures have evolved considerably opening new opportunities for code improvements, based on parallelism and use of compute accelerators.

        In this talk we present the R&D activities to cope with the new HEP computing challenges, taking place in the context of the Geant4 simulation toolkit. We first discuss the general scope and plan of this initiative and we introduce the different directions that are being explored with the potential benefits they can bring. The second part is focused on a few concrete examples of the R&D projects, like the use of tasking-based parallelism with possible off-load to GPUs, introduction of vectorization at different stages of the simulation or implementation of ‘per volume’-specialized geometry navigators. We discuss the technical details of the different prototype implementations. In conclusion, our first results in those different areas are reported and the plans for the near future are presented.

        Speaker: Andrei Gheata (CERN)
    • 15:30 16:30
      Posters: A Hall F

      Hall F

      Adelaide Convention Centre

      • 15:30
        Evaluation of Linux distributions for SoC devices on custom electronics in the CMS Network 15m

        System on Chip (SoC) devices have become popular for custom electronics HEP boards. Advantages include the tight integration of FPGA logic with CPU, and the option for having relatively powerful CPUs, with the potential of running a fully fledged operating system.

        In the CMS trigger and data acquisition system, there are already a small number of back-end electronics boards with Xilinx Zync SoCs in use since 2015 (LHC run-2). These are stand-alone installations. For the High Luminosity phase of the LHC starting around 2026, entirely new CMS back-end electronics is being developed. It is expected that SoC devices will be used at large scale (order of 1000) comparable to the number of High Level Trigger (HLT) nodes today, but with diverse use cases, hardware types, and capabilities (memory, cpu power).
        This large scale will pose challenges for their integration in the experiment network, system administration services and overall configuration management. Issues include the time distribution, IP/name distribution (DHCP or other), remote system logs, read-only or read-write root file systems, NFS mounted root or application file systems, local or network system boot, and configuration management of devices on various linux distributions. Furthermore, with the emergence of more powerful CPUs it will be interesting to see how much of the data acquisition control and monitoring software could or should be deployed on those devices compared to server PCs.

        We have evaluated a number of Linux distributions (Yocto, PetalLinux, ArchLinux, CentOS), addressing the complexity of building a distribution, the requirements on hardware resources, and the characteristics for network and sysadmin integration.

        Speaker: Marc Dobson (CERN)
      • 15:45
        SDN for End-to-End Networking at Exascale 15m

        The Caltech team in collaboration with network, computer science, and HEP partners at the DOE laboratories and universities, building smart network services ("The Software-defined network for End-to-end Networked Science at Exascale (SENSE) research project") to accelerate scientific discovery.

        The overarching goal of SENSE is to enable National Labs and universities to request and provision end-to-end intelligent network services for their application workflows leveraging SDN capabilities. The project’s architecture, models, and demonstrated prototype define the mechanisms needed to dynamically build end-to-end virtual guaranteed networks across administrative domains, with no manual intervention. In addition, a highly intuitive ‘intent’ based interface, as defined by the project, allows applications to express their high-level service requirements, and an intelligent, scalable model-based software orchestrator converts that intent into appropriate network services, configured across multiple types of devices.

        The overarching goal of SENSE is to enable National Labs and universities to request and provision end-to-end intelligent network services for their application workflows leveraging SDN capabilities.

        In this paper, we will present system's architecture and it's components, first results of dynamic network resource provisioning and Quality of Service for data transfers using FTS3 and other transfer protocols, like GridFTP, XRootD, FDT.

        Speaker: Jean-Roch Vlimant (California Institute of Technology (US))
      • 16:00
        Partial wave analysis with OpenAcc 15m

        Partial wave analysis is an important tool in hadron physics. Large data sets from the experiments in high precision frontier require high computational power. To utilize GPU cluster and the resource of supercomputers with various types of the accelerator, we implement a software framework for partial wave analysis using OpenAcc, OpenAccPWA. OpenAccPWA provides convenient approaches for exposing parallelism in the code and excellent support for a large amount of existing CPU-based codes of partial wave amplitudes. It can avoid a heavy workload of code migration from CPU to GPU.

        This poster will briefly introduce the software framework and performance of OpenAccPWA.

        Speaker: Yanjia Xiao
      • 16:10
        A culture shift: transforming learning at CERN 20m

        To accomplish its mission, the European Centre for Nuclear Research (CERN, Switzerland) is committed to the continuous development of its personnel through a systematic and sustained learning culture, that aims at keeping the knowledge and competences of the personnel in line with the evolving needs of the Organisation.
        With this goal in mind, CERN supports learning in its broadest sense and promotes a variety of learning methods. Over the last few years, CERN has focused its efforts on expanding the learning opportunities of its personnel via newly available software and e-learning tools and methodologies, thereby bringing a shift in the learning culture of the organisation. In September 2018, CERN launched a new Learning Management System centralizing all learning opportunities in a single platform, the ‘CERN Learning Hub’. In addition, new e-learning resources are now widely available to the personnel, including customized internally created e-learnings, an e-library, a commercial e-learning platform for self-paced learning and online surveys (180/360 feedback tools for CERN manager and leaders).
        This paper presents the experience gained by CERN in testing and adopting these new e-learning technologies and discusses the future vision for CERN.

        Speaker: Sebastian Bukowiec (CERN)
      • 16:15
        A Faster, More Accessible RooFit 15m

        RooFit and RooStats, the toolkits for statistical modelling in ROOT, are used in most searches and measurements at the Large Hadron Collider, as well as B factories. The large datasets to be collected in Run 3 will enable measurements with higher precision, but will require faster data processing to keep fitting times stable.
        In this talk, a redesign of RooFit’s internal dataflow will be presented. Cache locality and data loading are improved, and batches of data are processed with vectorised SIMD computations. This improves RooFit’s single-thread performance by several orders. In conjunction with multiple workers, this will allow to fit the larger datasets of Run 3 in the same time or faster than today’s fits.
        RooFit’s interfaces will further be extended to be more accessible both from C++ and Python to improve interoperability and ease of use.

        Speaker: Stephan Hageboeck (CERN)
      • 16:15
        A Functional Declarative Analysis Language in Python 15m

        Based on work in the ROOTLINQ project, we’ve re-written a functional declarative analysis language in Python. With a declarative language, the physicist specifies what they want to do with the data, rather than how they want to do it. Then the system translates the intent into actions. Using declarative languages would have numerous benefits for the LHC community, ranging from analysis preservation that goes beyond the lifetimes of experiments or analysis software, to facilitating the abstraction, design, validation, combination, interpretation and overall communication of the contents of LHC analyses. This talk focuses on an ongoing effort to define an analysis language based on queries, designed to loop over structured data including a complete set of unambiguous operations. This project has several implementation goals: 1) Design a syntax that matches how physicists think about event data, 2) Run on different back-end formats, including binary data (xAOD’s from ATLAS, for example), flat TTree’s using RDataFrame, and columnar data in python. This work will further help to understand the differences between Analysis Languages and Data Query Languages in HEP, how hard it is to translate data manipulation from a row-wise-centric layout to a column-wise-centric layout, and, finally, to scale from a small laptop-like environment to a larger cluster. The system currently has all three backends implemented to varying degrees and is being used in a full Run 2 analysis in ATLAS. The plans, goals, design, progress, and pitfalls will be described in this presentation.

        Speaker: Emma Torro Pastor (University of Washington (US))
      • 16:15
        A gateway between Gitlab CI and DIRAC 15m

        The Gitlab continuous integration system (http://gitlab.com) is an invaluable tool for software developer to test and validate their software. LHCb analysts have also been using it to validate physics software tools and data analysis scripts, but this usage faced issues differing from standard software testing, as it requires significant amount of CPU resources and credentials to access physics data. This paper presents the Gitlab CI to DIRAC gateway, a tool that runs Gitlab CI jobs within the LHCb grid system (LHCbDirac) therefore bridging the gap between the Gitlab jobs and the CPU and disk resources provided to the experiment.

        Speaker: Ben Couturier (CERN)
      • 16:15
        A global track finding algorithm for CGEM +DC with Hough Transform 15m

        A present-day detection system for charged tracks in particle physics experiments is typically composed of two or more types of detectors. Then global track finding with these sub-detectors is one important topic. This contribution is to describe a global track finding algorithm with Hough Transform for a detection system consist of a Cylindrical-Gas-Electron-Multiplier (CGEM) and a Drift Chamber (DC). The detailed Hough Transform of the hits detected by CGEM and DC, the optimization of the binning of Hough maps, the global track fitting, the iterative way to determine tracks and some results with simulated samples are going to be presented.

        Speaker: Dr Linghui Wu (Institute of High Energy Physics)
      • 16:15
        A Lightweight Door into Non-grid Sites 15m

        The Open Science Grid (OSG) provides a common service for resource providers and scientific institutions, and supports sciences such as High Energy Physics, Structural Biology, and other community sciences. As scientific frontiers expand, so does the need for resources to analyze new data. For example, high energy physics (LHC) sciences foresee an exponential growth in the amount of data collected, which comes with corresponding growth in the need for computing resources. Allowing resource providers an easy way to share their resources is paramount to ensure the grow of resources available to scientists.

        In this context, the OSG Hosted CE initiative provides site administrator a way to reduce the effort needed to install and maintain a Compute Element (CE), and represents a solution for sites who do not have the effort and expertise to run their own Grid middleware. An HTCondor Compute Element is installed on a remote VM at UChicago for each site that joins the Hosted CE initiative. The hardware/software stack is maintained by OSG Operations staff in a homogeneus and automated way, providing a reduction in the overall operational effort needed to maintain the CEs: one single organization does it in an uniform way, instead of each single resource provider doing it in their own way. Currently, more than 20 institutions joined the Hosted CE initiative. This contribution discusses the technical details behind a Hosted CE installation, highlighting key strenghts and common pitfalls, and outlining future plans to further reduce operational experience.

        Speaker: Igor Sfiligoi (UCSD)
      • 16:15
        An ARM cluster for running CMSSW jobs 15m

        The ARM platform extends from the mobile phone area to development board computers and servers. It could be that in the future the importance of the ARM platform will increase if new more powerful (server) boards are released. For this reason CMSSW has previously been ported to ARM in earlier work.

        The CMS software is deployed using CVMFS and the jobs are run inside Singularity containers. Some ARM aarch64 CMSSW releases are available in CVMFS for testing and development. In this work CVMFS and Singularity have been compiled and installed on an ARM cluster and the aarch64 CMSSW releases in CVMFS have been used. We report on our experiences with this ARM cluster for CMSSW jobs.

        Speaker: Tomas Lindén (Helsinki Institute of Physics (FI))
      • 16:15
        Applying OSiRIS NMAL to Network Slices on SLATE 15m

        We will present techniques developed in collaboration with the OSiRIS project (NSF Award #1541335, UM, IU, MSU and WSU) and SLATE (NSF Award #1724821) for orchestrating software defined network slices with a goal of building reproducible and reliable computer networks for large data collaborations. With this project we have explored methods of utilizing passive and active measurements to build a carefully curated model of the network. We will show that by then using such a model, we can dynamically and programmatically alter network and host configuration to effectively respond to changing network conditions.

        As part of our presentation, we will show how SLATE, operating over a slice of the Internet2 network, provides a container focused platform for running a Network Management Abstraction Layer (NMAL), allowing us to control applications in a reliable and reproducible way. This presentation will demonstrate how NMAL tracks live network topological and performance statistics on an Internet2 slice with SLATE-enabled hosts to enact traffic engineering and container placement decisions in order to optimize network behavior based on user defined profiles. We will conclude by discussing the future of this work and our plans for using it to support science activities in production.

        Speaker: Shawn Mc Kee (University of Michigan (US))
      • 16:15
        BAT.jl – Upgrading the Bayesian Analysis Toolkit 15m

        BAT.jl, the Julia version of the Bayesian Analysis Toolkit, is a software package which is designed to help solve statistical problems encountered in Bayesian inference. Typical examples are the extraction of the values of the free parameters of a model, the comparison of different models in the light of a given data set, and the test of the validity of a model to represent the data set at hand. BAT.jl is based on Bayes’ Theorem and it is realized with the use of different algorithms. These give access to the full posterior probability distribution, and they enable parameter estimation, limit setting and uncertainty propagation.
        BAT.jl is implemented in Julia and allows for a flexible definition of mathematical models and applications while keeping in mind the reliability and speed requirements of the numerical operations. It provides implementations (or links to implementations) of algorithms for sampling, optimization and integration. While predefined models exist for standard cases, such as simple counting experiments, binomial problems or Gaussian models, its full strength lies in the analysis of complex and high-dimensional models often encountered in high energy and nuclear physics.
        BAT.jl is a completely re-written code based on the original BAT code written in C++ . There is no backward compatibility whatsoever, but the spirit is the same: providing a tool for Bayesian computations of complex models.
        The poster will summarize the current status of the BAT.jl project and highlight the challenges faced in the fields of high energy and nuclear physics.

        Speakers: Cornelius Grunwald (Technische Universitaet Dortmund (DE)), Cornelius Grunwald (TU Dortmund)
      • 16:15
        Calibration and Performance of the CMS Electromagnetic Calorimeter in LHC Run2 15m

        Many physics analyses using the Compact Muon Solenoid (CMS) detector at the LHC require accurate, high resolution electron and photon energy measurements. Excellent energy resolution is crucial for studies of Higgs boson decays with electromagnetic particles in the final state, as well as searches for very high mass resonances decaying to energetic photons or electrons. The CMS electromagnetic calorimeter (ECAL) is a fundamental instrument for these analyses and its energy resolution is crucial for the Higgs boson mass measurement. Recently the energy response of the calorimeter has been precisely calibrated exploiting the full Run2 data, aiming at a legacy reprocessing of the data. A dedicated calibration of each detector channel has been performed with physics events exploiting electrons from W and Z boson decays, photons from pi0/eta decays, and from the azimuthally symmetric energy distribution of minimum bias events. This talk presents the calibration strategies that have been implemented and the excellent performance achieved by the CMS ECAL with the ultimate calibration of Run II data, in terms of energy scale stability and energy resolution.

        Speaker: Chiara Ilaria Rovelli (Sapienza Universita e INFN, Roma I (IT))
      • 16:15
        Code health in EOS: Improving test infrastructure and overall service quality 15m

        During the last few years, the EOS distributed storage system at CERN has seen a steady increase in use, both in terms of traffic volume as well as sheer amount of stored data.

        This has brought the unwelcome side effect of stretching the EOS software stack to its design constraints, resulting in frequent user-facing issues and occasional downtime of critical services.

        In this paper, we discuss the challenges of adapting the software to meet the increasing demands, while at the same time preserving functionality without breaking existing features or introducing new bugs. We document our efforts in modernizing and stabilizing the codebase, through the refactoring of legacy code, introduction of widespread unit testing, as well as leveraging kubernetes to build a comprehensive test orchestration framework capable of stressing every aspect of an EOS installation, with the goal of discovering bottlenecks and instabilities before they reach production.

        Speaker: Andreas Joachim Peters (CERN)
      • 16:15
        Compass SPMD, a SPMD vectorized tracking algorithm 15m

        The LHCb detector will be upgraded in 2021, where the hardware-level trigger will be replaced by a High Level Trigger 1 software trigger that needs to process the full 30 MHz data-collision rate. As part of the efforts to create a GPU High Level Trigger 1, tracking algorithms need to be optimized for SIMD architectures in order to achieve high-throughput. We present a SPMD (Single Program, Multiple Data) version of Compass, a tracking algorithm optimized for SIMD architectures, vectorized using the Intel SPMD Program Compiler. This compiler and model allows to execute program instances in parallel, and allows to use exploit the SIMD lanes of CPUs using GPU-like source code, without the need of low-level details knowledge. It is able to target different vector widths, vector instructions sets and combine different levels of parallelism. We design the algorithm focusing on highly parallel architectures in mind, minimizing divergence and memory footprint while creating a data-oriented algorithm that is efficient for SIMD architectures. We vectorize the algorithm using the SPMD programming model, preserving the algorithm design and delivering the same physics efficiency as its GPU counterpart. We study the physics performance and throughput of the algorithm. We discuss the impact with different vector widths and instructions sets and compare it with the GPU implementation.

        Speaker: Placido Fernandez Declara (University Carlos III (ES))
      • 16:15
        Computing for the general public at CERN’s Science Gateway 15m

        CERN is launching the Science Gateway, a new scientific education and outreach centre targeting the general public of all ages. Construction is planned to start in 2020 and to be completed in 2022. In addition to Physics exhibits, the Science Gateway will include immersive, hands-on activities that explore Computer Science and Technology. This poster will present the methodology used to generate the content for the computing installations and showcase the ongoing progress towards bringing the concepts to life.

        Speaker: Hannah Short (CERN)
      • 16:15
        Conditions Databases at FNAL 15m

        Conditions databases is an important class of database applications where the database is used
        to record the state of a set of quantities as a function of observation time.
        Conditions databases are used in Hight Energy Physics to record the state of
        the detector apparatus during data taking, and then to use the data during
        the event reconstruction and analysis phases.

        At FNAL, we have a set of 3 different conditions database products, Minerva Conditions DB, ConDB and UConDB, which cover the whole range of the use cases presented by the FNAL experiments. These products have common features such as conditions data representation model, data version control, ability to restore the database to a previous state, scalable web service data access interface. In the paper, we will present the common features of the products, common solutions used to build them and also the differences between the products and their target use cases.

        Speaker: Eric Vaandering (Fermi National Accelerator Lab. (US))
      • 16:15
        Data Management and User Data Portal at CSNS 15m

        China Spallation Neutron Source (CSNS) is a large science facility, and it is public available to researchers from all over the world. The data platform of CSNS is aimed for diverse data and computing supports, the design philosophy behind is data safety, big-data sharing, and user convenience.
        In order to manage scientific data, a metadata catalogue based on ICAT is built to manage full life-time experiment metadata, from idea to publication. It is used as the data middleware, forms the basis of various data services. A multi-layered, distributed storage layout based on iRODS is adopted to store the data files, which enables the data virtualization and workflow automation. The digital object identifier (DOI) is applied to data itself to uniquely identify the data and to enable trace, interoperation and discovery of data.
        The web-based user data portal providers user with one-stop services for experiment information view, data search, retrieve, analysis and share in anytime and anywhere. Furthermore, a cloud analysis portal is developed based on Openstack, which enables user to utilize CSNS computing resources to handle data on demand.

        Speaker: Mr Ming Tang (Institute of High Energy Physics, Chinese Academy of Sciences)
      • 16:15
        Design principles of the Metadata Querying Language (MQL) implemented in the ATLAS Metadata Interface (AMI) ecosystem 15m

        ATLAS Metadata Interface (AMI) is a generic ecosystem for metadata aggregation, transformation and cataloging benefiting from about 20 years of feedback in the LHC context. This poster describes the design principles of the Metadata Querying Language (MQL) implemented in AMI, a metadata-oriented domain-specific language allowing to query databases without knowing the relation between tables. With this simplified yet generic grammar, MQL permits writing complex queries much more simply than Structured Query Language (SQL). The poster describes how AMI compiles MQL into SQL queries using the underlying table relations graph automatically extracted through a reflexion mechanism.

        Speaker: Jerome Odier (LPSC/CNRS (Grenoble, FR))
      • 16:15
        Detector Construction Application for CMS Ph2 Detectors 15m

        During the third long shutdown of the CERN Large Hadron Collider, the CMS Detector will undergo a major upgrade to prepare for Phase-2 of the CMS physics program, starting around 2026. Upgrade projects will replace or improve detector systems to provide the necessary physics performance under the challenging conditions of high luminosity at the HL-LHC. Among other upgrades, the new CMS Silicon-Tracker will substantially increase in the number of channels and will feature an improved spatial resolution. The new Endcap Calorimeter will allow measurement of the 3D topology of energy deposits in particle showers induced by incident electrons, photons and hadrons, as well as precise time-stamping of neutral particles down to low transverse momentum.

        Ph2 upgrade project collaborations consist of dozens of institutions, many participating in actual detector design, development, assembly and quality control (QC) testing. In terms of participating institutions and worldwide responsibilities Ph2 HGCAL and Outer Tracker projects are unprecedented. This raises a huge challenge for detector parts tracking, assembly and QC information bookkeeping.

        Detector Construction Application (DCA) is based on the universal database model which is capable of hosting information about different detectors assembly (construction), parts tracking between institutions and QC information. DCA consists of and maintains a number of tools for data upload (DB Loader), retrieval (Restful API), editing and analysis GUI. In this report we present the design and architecture of DCA which helps physicists and institutions to collaborate worldwide while building the next CMS detector.

        Speaker: Valdas Rapsevicius (Vilnius University (LT))
      • 16:15
        Distributed Caching in the WLCG 15m

        With the evolution of the WLCG towards opportunistic resource usage and cross-site data access, new challenges for data analysis have emerged in recent years. To enable performant data access without relying on static data locality, distributed caching aims at providing data locality dynamically. Recent work successfully employs various approaches for effective and coherent caching, from centrally managed approaches employing volatile storage to hybrid approaches orchestrating autonomous caches. However, due to the scale and use-case of the WLCG there is little prior work to assess the general applicability and scalability of these approaches.

        Building on our previous developments of coordinated, distributed caches at KIT, we have identified the primary challenge not in the technical implementation but the underlying logic and architecture. We have studied several key issues solved by all approaches in various ways: aggregation of meta-data, identification of viable data, coherence of cache contents, and integration of temporary caches. Monitoring data from XRootD storage and HTCondor batch systems from both our Tier 1 and Tier 3 infrastructure provides realistic usage characteristics. This allows us to assess both the use cases of well-defined user jobs, as well as late-bound anonymous pilot jobs.

        In this contribution, we present our findings on the implications of different architectures for distributed caching depending on the targeted use-case.

        Speaker: Max Fischer (Karlsruhe Institute of Technology)
      • 16:15
        End-to-end Deep Learning Fast Simulation Framework 15m

        To address the increase in computational costs and speed requirements for simulation related to the higher luminosity and energy of future accelerators, a number of Fast Simulation tools based on Deep Learning (DL) procedures have been developed. We discuss the features and implementation of an end-to-end framework which integrates DL simulation methods with an existing Full Simulations toolkit (Geant4). We give a description of the key concepts and challenges in developing a production environment level Simulation framework based on Deep Neural Network (DNN) models designed for High Energy Physics (HEP) problem domain and trained on HEP data. We discuss, data generation (simplified calorimeters simulations obtained with the Geant4 toolkit) and processing, DNN architecture evaluation procedures and API integration. We address the challenge of distributional shifts in the input data (dependent on calorimeter type) and evaluate the response of trained networks and propose a general framework for physics validation of DL models in HEP.

        Speaker: Ioana Ifrim (CERN)
      • 16:15
        Enhancements in Functionality of the Interactive Visual Explorer for ATLAS Computing Metadata 15m

        The development of the Interactive Visual Explorer (InVEx), a visual analytics tool for ATLAS computing metadata, includes research of various approaches for data handling both on server and on client sides. InVEx is implemented as a web-based application which aims at the enhancing of analytical and visualization capabilities of the existing monitoring tools, and facilitate the process of data analysis with the interactivity and human supervision. The development of InVEx started with the implementation of a 3-dimensional interactive tool for cluster analysis (for the k-means and DBSCAN algorithms), and its further evolvement is closely linked to the needs of ATLAS computing experts, providing metadata analysis to ensure the stability and efficiency of the distributed computing environment functionality. In the process of the integration of InVEx with ATLAS computing metadata sources we faced two main challenges: 1) big data volumes needed to be analyzed in the real time mode (as an example, one ATLAS computing task may contain tens of thousands jobs, each having over two hundred of different parameters), and 2) machine learning clustering algorithms alone are not sufficient for visual cluster analysis - the ability of user-defined clusterization/grouping (by nominal or ordinal parameters) should be added, to make the process of data analysis more manageable.
        The current work is focused on the architecture enhancements of the InVEx application. First, we will describe the user-manageable data preparation method for cluster analysis. Then, we will present the Level-of-Detail method for the interactive visual data analysis. Beginning with the low detalization, when all data are grouped (by clusterization algorithms or by parameters) and aggregated, we provide users with means to look deeply into this data, incrementally increasing the level of detalization. And finally, the development of data storage format for InVEx is adapted for the Level-of-Detail method to keep all stages of data derivation sequence.

        Speaker: Siarhei Padolski (BNL)
      • 16:15
        Erratic server behavior detection using machine learning on streams of monitoring data 15m

        With the explosion of the number of distributed applications, a new dynamic server environment emerged grouping servers into clusters, utilization of which depends on the current demand for the application. To provide reliable and smooth services it is crucial to detect and fix possible erratic behavior of individual servers in these clusters. Use of standard techniques for this purpose requires manual work and delivers suboptimal results.

        Using only application agnostic monitoring metrics our machine learning based method analyzes the recent performance of the inspected server as well as the state of the rest of the cluster, thus checking not only the behavior of the single server, but the load on the whole distributed application as well. We have implemented our method in a Spark job running in the CERN MONIT infrastructure.

        In this contribution we present results of testing multiple machine learning algorithms and preprocessing techniques to identify the servers erratic behavior. We also discuss the challenges of deploying our new method into production.

        Speaker: Martin Adam (Acad. of Sciences of the Czech Rep. (CZ))
      • 16:15
        Evaluation of a new visualization and analytics solution for slow control data for large scale experiments 15m

        Large experiments in high energy physics require efficient and scalable monitoring solutions to digest data of the detector control system. Plotting multiple graphs in the slow control system and extracting historical data for long time periods are resource intensive tasks. The proposed solution leverages the new virtualization, data analytics and visualization technologies such as InfluxDB time-series database (DB) for faster access large scale data, Grafana to visualize time-series data and an OpenShift container platform to automate build, deployment, and management of application. The monitoring service runs separately from the control system thus reduces a workload on the control system computing resources. As an example, a test version of the new monitoring was applied to the ATLAS Tile Calorimeter using the CERN Cloud Process as a Service platform. Many dashboards in Grafana have been created to monitor and analyze the behavior the High Voltage distribution system. They visualize not only values measured by the control system, but also run information and analytics data (difference, deviation, etc.). The new monitoring with a feature-rich visualization, filtering possibilities and analytics tools allows to extend detector control and monitoring capabilities and can help experts working on large scale experiments.

        Speaker: Laura Sargsyan (A.Alikhanyan National Science Laboratory (AM))
      • 16:15
        Evolution of the Data Quality Monitoring and Prompt Processing System in the protoDUNE-SP experiment 15m

        The DUNE Collaboration has successfully implemented and currently operates
        an experimental program based at CERN which includes a beam test and an extended
        cosmic ray run of two large-scale prototypes of the DUNE Far Detector. The volume of data already collected by the protoDUNE-SP (the single-phase Liquid Argon TPC prototype) amounts to approximately 3PB and the sustained rate of data sent to mass storage is of the order of O(100) MB/s. In addition to this data being committed to mass storage and processed in the Grid environment, a small fraction of it is captured by the protoDUNE Prompt Processing System (p3s). The p3s is optimized for continuous low-latency calculation of the vital detector metrics and parameters as well as the output rendered as event display images. This system is the platform for Data Quality Monitoring in protoDUNE-SP and has served a crucial role starting from the commissioning of the apparatus and throughout its operation in 2018-2019, which continues at the time of writing. We present our experience in operating the system in the CERN environment, along with work currently underway to make the system more scalable, resilient and to simplify system recovery procedures in preparation for the second beam run of protoDUNE-SP foreseen after the Long Shutdown 2 of the LHC.

        Speaker: Maxim Potekhin (Brookhaven National Laboratory (US))
      • 16:15
        Evolution of the load balancing server 15m

        The Load Balance Service at CERN handles more that 400 aliases,
        distributed over more than 2000 nodes. After being in production for
        more than thirteen years, it has been going through a mayor redesign
        over the last two years. Last year, the server part got reimplemented in
        golang, taking advantage of the concurrency features offered by the
        language to improve the scaling of the system. This year, the client
        side and the communication between the client and the server have been
        the main focus. First of all, the client part of the Load Balance Service has
        been rewritten in golang. On top of that, it was evaluated to move from a model
        where the server probes the clients, to a model where the clients
        announce their states. This has a quite a lot of implications in the
        frequency of the updates, the security model and the deployment of the
        service. Finally, it was also evaluated to offer
        'HAProxy as a Service', taking feedback on the client members using the same probes currently used for the DNS load balancing.

        All the components used by the Load Balance Service are open source, and
        they can be deployed at other places

        Speaker: Pablo Saiz (CERN)
      • 16:15
        Featherweight Communication Between The Tasks For Spark 15m

        ABSTRACT

        Apache Spark is a splendid framework for big data analysis nowadays. A Spark application can be divided into some jobs which are triggered by an action of RDD, then the jobs will be divided into stages by the DAGScheduler, after these processes, we will get the task which is a unit of work within a stage, corresponding to one RDD partition.

        Task is the smallest unit when Spark executes the application. However there is no communication between tasks in the current Spark framework. This article discusses the reasons why we need to extend Spark by compiling an API which can offer featherweight communication between tasks. This API won’t break current communication mode and can be portable to standard Spark installations. At last we give some examples to explain how to address the specified situation in the high energy physics field.

        Keywords
        Spark, featherweight communication,task, high energy physics

        Speaker: Mr YI WANG
      • 16:15
        Federated User Account Management 15m

        BNL SDCC(Sentific Data and Computing Center) recently enabled centralized identity management solution. With SSO authentication process being enabled to cross multiple IT systems or organizations including federated login access via CILogon InCommon. With the combination of MFA/DUO to meet security standards for various application & services such as Jupyterhub/Invenio provided to the communities. CoManage (Cloud) and FreeIPA / Keycloak (local) are utilized to provided complex authorization for authenticated users.This talk will focus on technical overviews and strategies to tackle the challenges/obstacles in our facility.

        Speaker: Shigeki Misawa (BNL)
      • 16:15
        Fine Tuning of Generative Models for the Fast Simulation 15m

        The goal to obtain more precise physics results in current collider experiments drives the plans to significantly increase the instantaneous luminosity collected by the experiments. The increasing complexity of the events due to the resulting increased pileup requires new approaches to triggering, reconstruction, analysis,
        and event simulation. The last task brings to a critical problem: generating the significantly higher amount of Monte Carlo (MC) data, required for analysis of the data collected at higher collider luminosity, without a drastic increase in computing resources requires a significant speedup of the simulation algorithms.
        The largest part of computer resources in simulation is currently spent in the detailed GEANT modeling of particles interacting with the material of the experimental apparatus, in particular, the shower development in electromagnetic and hadronic calorimeters.
        To accelerate these computations approaches based on methods of sample creation by generative models may be used.
        However, those properties and behaviours that are most important for the validity of generated data are sometimes not those for which the model mostly cares.
        In this contribution, we discuss possible approaches to making model sensitive to those properties that are most important for generated objects from the physics perspective.
        Practical cases will be presented to illustrate these approaches.

        Speaker: Fedor Ratnikov (Yandex School of Data Analysis (RU))
      • 16:15
        Generative Adversarial Network for Background Generation in the KLM subsystem at Belle II 15m

        The second-generation Belle II experiment at the SuperKEKB colliding-beam accelerator in Japan searches for new-physics signatures and studies the behaviour of heavy quarks and leptons produced in electron-positron collisions. The KLM (K-long and Muon) subsystem of Belle II identifies long-lived neutral kaons via hadronic-shower byproducts and muons via their undeflected penetration through dense matter. GEANT4-based Monte Carlo simulations of physics processes in Belle II are supplemented by the overlay of additional hits on each event, sampled from a curated library of background events. We describe the proposed use of a generative adversarial network to construct dynamically these background overlay hits in the KLM and compare the GAN's performance relative to curated-library approach.

        Speaker: Leo Piilonen (Virginia Tech)
      • 16:15
        GTS - Garfield-based Triple-GEM Simulator 15m

        Triple-GEM detectors are gaseous devices used in high energy physics to measure the path of the particles which cross them. The characterisation of triple GEM detectors and the estimation of the performance for real data experiments require a complete comprehension of the mechanisms which transform the passage of one particle in the detector into electric signals, and dedicated MonteCarlo simulations are needed. In this work we will describe GTS (Garfield-based Triple-gem Simulator), a MonteCarlo code which has been developed to simulate the detector response from the passage of particles inside triple GEMs. The software takes into account the processes from the primary ionization up to the signal formation, e.g. avalanche multiplication and the effects of the diffusion on the signal in the gas volume. The software uses a parametrization of the variables of interest to reproduce the detector response based on Garfield, a well known software in literature that already performs this kind of simulation, but with a high consumption of CPU time: GTS reduces significantly the computation time. In addition to the detector response, the simulation of the APV-25 electronics is implemented and the output is used to reconstruct the particle position with the Charge Centroid (CC) and the micro-Time Projection Chamber (µTPC) methods, for the first time in a triple-GEM simulator. A comparison of the simulated performance and the one collected in test beams is used to tuned the parameters used in GTS. Results in different conditions of magnetic field, high voltage settings and incident angle will be shown.

        Speaker: Riccardo Farinelli (Universita e INFN, Ferrara (IT))
      • 16:15
        Improvements in the NOvA Detector Simulation based on JINR stand measurements 15m

        NOvA is a long-baseline neutrino experiment aiming to study neutrino oscillation phenomenon in the muon neutrino beam from complex NuMI at Fermilab (USA). Two identical detectors have been built to measure the initial neutrino flux spectra at the near site and the oscillated one at a 810 km distance, which significantly reduces many systematic uncertainties. To improve electron neutrino and neutral current interaction separation, the detector is constructed as a finely segmented structure filled with liquid scintillator. Charged particles lose their energy in the detector materials, producing light signal in a cell which are recorded by readout electronics. The simulation models this using the following chain: a parameterized front-end simulation converts all energy deposits in active material into scintillation light, the scintillation light is transported through an optical fiber to an avalanche photodiode, and the readout electronics simulation models the shaping, digitization, and triggering on the response of the photodiode. Two test stands have been built in JINR (Dubna, Russia) to measure the proton light response of NOvA scintillator and the electronic signal shaping of the NOvA front-end-board. The parameters measured using these test stands have been implemented in the custom NOvA simulation chain. In additional a new data-driven tuning has been performed for a model of the Cherenkov radiation in the scintillator.

        Speaker: Oleg Samoylov (Joint Institute for Nuclear Research (RU))
      • 16:15
        Infrsastracture ACI fabric based on EVPN MPBGP and TRILL data transfer protocols for Tier 1 and Tier 2 data centers 15m

        This paper presents the network architecture of the TIER 1 data center at JINR using the modern multichannel data transfer protocol TRILL. The obtained experimental data folow our activity to further study the nature of traffic distribution in redundant topologies. There are several questions. How the distribution of packet data occurs on four (or more) equivalent routes? What happens when the download on one of the four communication channels reaches peak values?
        The paper presents a future data center topology, all network elements of which will be interconnected in the Virtual Cluster Switching (VCS) fabric. Such architecture will allow building highly reliable, mobile, multi-port network segments.
        Future the network segment project is being designed using EVPN MP-BGP (Ethernet Virtual Private Networks Multiprotocol Border Gateway Protocol) technology in conjunction with an external Application Centric Infrastructure (ACI) controller based on VXLAN technology using a multicast broadcast domain.
        Today, technology analysis shows that this is the most acceptable choice, tested in several R @ D divisions of the world's largest vendors.
        Finally, the results of the comparison of Virtual Cluster Switching (VCS) fabric based on TRILL and EVPN MP-BGP are predicted.

        Speaker: Andrey Baginyan (Joint Institute for Nuclear Research (RU))
      • 16:15
        Installing and administrating the ATLAS Metadata Interface (AMI) ecosystem 15m

        ATLAS Metadata Interface (AMI) is a generic ecosystem for metadata aggregation, transformation and cataloging. Benefiting from about 20 years of feedback in the LHC context, the second major version was released in 2018. This poster describes how to install and administrate AMI version 2. A particular focus is given to the registration of existing databases in AMI, the adding of additional metadata and the generation of high level HTML 5 search interfaces using a dedicated wizard. Finally, this poster shows how to extend AMI by writing server-side Java commands, tasks and client-side HTML 5 applications.

        Speaker: Dr Jerome Odier (LPSC/CNRS (Grenoble, FR))
      • 16:15
        ITER data infrastructure and mapping of experimental data and machine description into a shared data format. 15m

        J. Hollocombe[1]​ ​, Eurofusion ​WPISA CPT​, Eurofusion WPCD
        1. UKAEA, ​Culham Science Centre, OX14 3DB

        The ITER Data Model has been created to allow for a common data representation to be used by codes simulating ITER relevant physics. A suite of tools has been created to leverage this data structure called the Integrated Modelling & Analysis Suite (IMAS). As part of an exercise to utilise these tools for existing codes on existing experiments data mappings have been created to transform signal and machine description data from each machine's respective data archive and file formats into a common IDS repository. This has allowed for disparate data sources to be collated and codes to be run across different experiments.

        This poster will present the current state of the ITER data structure and related data access technologies (IMAS) as well as describe the work that has been done under Eurofusion WPCD (wpcd-workflows.github.io) to allow for the transformation and access of the data archives of existing experiments via the ITER architecture. It will display some of physics codes that have been updated to allow them to be run on data from any of these mapped data and the results of these simulations.

        Speaker: Andrew Lahiff
      • 16:15
        JUNO Calibration Complex and its Simulation 15m

        The Jiangmen Underground Neutrino Observatory (JUNO) is designed to primarily measure the neutrino mass hierarchy. The JUNO central detector (CD) would be the world largest liquid scintillator (LS) detector with an unprecedented energy resolution of 3\%/\sqrt{E(MeV)} and a superior energy nonlinearity better than 1%. A calibration complex, including Cable Loop System (CLS), Guide Tube Calibration System (GTCS), Auto Calibration Unit (ACU), and Remotely Operated Vehicle (ROV), is introduced with deploying multiple radioactive sources in various locations inside/outside of the CD to achieve this challenging calibration goal. The design and strategy of the JUNO calibration system had been optimized based on full Monte Carlo simulation results. The energy response could be described by fitting data from ACU, CLS and GTCS (or ROV) at given calibration points. And a concise spline function is utilized to predict a "blank" energy region and a correction function is applied to the uniformity of energy response. This talk will present details of the JUNO calibration complex and simulation results which help achieve an excellent energy resolution of 2.98% for 1.022 MeV uniformly distributed positrons with ~0.04% bias.

        Speaker: Prof. Qingmin Zhang (Xi'an Jiaotong University)
      • 16:15
        Jupyter-based service for JUNO analysis 15m

        The JUNO (Jiangmen Underground Neutrino Observatory) is designed to determine the neutrino mass hierarchy and precisely measure oscillation parameters. The JUNO central detector is a 20 kt spherical volume of liquid scintillator (LS) with 35m diameter instrumented with 18,000 20-inch photomultiplier tubes (PMTs). Neutrinos are captured by protons of the target via the inverse beta decay reaction which produces a positron and a neutron. The time correlation between the prompt signal from the annihilation and ionization of the positron and the delayed signal from the gamma rays produced by the neutron capture is the crucial feature that makes it possible to detect neutrinos. Analysis of time correlations to separate neutrino candidates from background sources of coincident signals is crucial to JUNO.

        This contribution presents the design and implementation of a Jupyter-based web client and server that facilitates the JUNO correlation analysis using a distributed processing middleware developed to provide a uniform interface to event data from a cluster of local nodes. Jupyter is an open source project providing a framework for interactive analysis, backend services and communication infrastructure. Selection criteria are defined within the Jupyter web client. The middleware provides a unified view of events in the correct event order using the resources of multiple cluster nodes. At initialization of each node, a list of event files in ROOT format are assigned to the node and registered into the middleware, which subsequently automatically loads and caches events in memory as needed. The performance advantages of distributed approach are presented.

        Speaker: Weidong Li (IHEP, Beijing)
      • 16:15
        Light-weight Grid Computing for Small Group Use Cases 15m

        In modern physics experiments, data analysis need considerable computing capacity. Computing resources of a single site are often limited and distributed computing is often inexpensive and flexible. While several large-scale grid solutions exist, for example DiRAC (Distributed Infrastructure with Remote Agent Control), there are few schemes devoted to solve the problem at small-scale. For the cases when light-weight grid computing is more desirable, our project provides a performant solution in connecting and managing the distributive computing resources of a small group. The components are all freely available as official Debian packages.

        In our project, an interior gateway routing protocol was deployed over a mesh overlay network spanning over several remote sites, so that servers can communicate with each other with high security and reliability. The Slurm workload manager distributed the computing tasks to the idle servers. The unified storage view was formed together using NFS (Network File System) Version 4.2. We have measured the aggregated computing power for pleasingly parallel workloads and found it to be comparable to summation of that of the individual ones. The computing resource is more scalable and flexible.

        Speaker: Mr Aiqiang ZHANG (+86 18810207912)
      • 16:15
        Machine Learning Pipelines for HEP Using Big Data Tools Applied to Improving Event Filtering 15m

        This work addresses key technological challenges in the preparation of data pipelines for machine learning and deep learning at scale of interest for HEP. A novel prototype to improve the event filtering system at LHC experiments, based on a classifier trained using deep neural networks has recently been proposed by T. Nguyen et al. https://arxiv.org/abs/1807.00083. This presentation covers how we implemented the data pipeline to train the neural network classifier using solutions from the Apache Spark and Big Data ecosystem, integrated with tools, software, and platforms common in the HEP environment. Data preparation and feature engineering make use of PySpark, Spark SQL and Python code run via Jupyter notebooks. We will discuss key integrations and libraries that make Apache Spark able to ingest data stored using ROOT and its integration EOS/XRootD protocol. The presentation will cover the neural network models used, defined using the Keras API, and how the models have been trained in a distributed fashion on Spark clusters using BigDL and Analytics Zoo. We will discuss the implementation, the results of the distributed training, and overall the lessons learned on using Big Data tools to implement an end-to-end ML pipeline.

        Speaker: Marco Zanetti (Universita e INFN, Padova (IT))
      • 16:15
        Modularization of the LHCb software environment and preparation for heterogeneous resources 15m

        The LHCb software stack has to be run in very different computing environments: the trigger farm at CERN, on the grid, on shared clusters, on software developer's desktops... The old model assumes the availability of CVMFS and relies on custom scripts (a.k.a LbScripts) to configure the environment to build and run the software. It lacks flexibility and does not allow, for example running in container and be very difficult to configure and run on non standard environments. This paper describes the steps taken to modularize this environment to allow for easier development and deployment (as standard python packages), but also added integration with container technology to better support non standard environments.

        Speaker: Marco Clemencic (CERN)
      • 16:15
        NANOAOD: a new compact event data format in CMS 15m

        The CMS Collaboration has recently commissioned a new compact data format, named NANOAOD, reducing the per-event compressed size to about 1-2 kB. This is achieved by retaining only high level information on physics objects, and aims at supporting a considerable fraction of CMS physics analyses with a ~20x reduction in disk storage needs. NANOAOD also facilitates the dissemination of analysis methods and the automation of standard workflows for deriving conditions and object calibrations. The latest developments of this project will be presented.

        Speaker: Karl Ehataht (National Institute of Chemical Physics and Biophysics (EE))
      • 16:15
        Network infrastructure for CZ Tier-2 Center 15m

        The Czech Tier-2 center hosted and operated by Institute of Physics of the Czech Academy o Sciences significantly upgraded external network connection in 2019. The older edge router Cisco 6509 provided several 10 Gbps connections via a 10 Gigabit Ethernet Fiber Module, from which 2 ports were used for external LHCONE conection, 1 port for generic internet traffic and 1 port to reach other Czech institutes hosting some servers of the distributed Tier-2. Three of these connections were realised via one single fiber leading to CESNET (the Czech NREN) routers using the DWDM (de)multiplexers.

        A new router Cisco Catalyst 9500 with a 100 Gbps ports enabled an upgrade of the connection to 100 Gbps over the same optical fiber link. However a much lower performance of this router caused problems with NAT. Some of the supported projects were less affected because of usage of IPv6 protocol. To overcome the smaller NAT capacities of the new router, we moved some worker nodes (WNs) to public IPv4 addresses. Not all WNs can use public IPv4 addresses due to their limited availability. Additional server was installed to provided NAT for remaining WNs.

        We will also present graphs with a typical traffic out and to the Tier-2 center to compare traffic via LHCONE and generic internet and size of transfers using IPv4 and IPv6 protocols.

        Speaker: Jiri Chudoba (Acad. of Sciences of the Czech Rep. (CZ))
      • 16:15
        New Developments in the VMC Project 15m

        Virtual Monte Carlo (VMC) provides a unified interface to different detector simulation transport engines such as GEANT3 and Geant4. Since recently, all VMC packages: the VMC core library, also included in ROOT, Geant3 and Geant4 VMC are distributed via the VMC Project GitHub organization. In addition to these VMC related packages, the VMC project also includes the Virtual Geometry Model (VGM), which is optionally used in Geant4 VMC for conversion between Geant4 and ROOT TGeo geometry models.
        In this contribution, we will present the new organization of the VMC project at GitHub and new developments in the VMC interface and the VMC packages. We will cover the introduction of the sensitive detector interface in the VMC core and both Geant3 and Geant4 VMC and the new Geant4-related developments.
        Geant4 VMC 3.0 with the integration of multithreading processing was presented at CHEP in 2015. In this presentation we will report on new features included in version 4.0 and beyond: the improved support for magnetic fields, the integration of fast simulation, Garfield++ physics, Geant4 transition radiation and monopole physics. Five new VMC examples demonstrating these new features, also used for tests, will be also discussed. Finally we will mention the work towards the code quality and improvements in testing, documentation and automated code formatting.
        The introduction of the multiple engine framework in the VMC packages and new developments in VGM will be covered in two separate contributions to this conference.

        Speaker: Ivana HRIVNACOVA (Institut de Physique Nucléaire (IPNO), Université Paris-Sud, CNRS-IN2P3, Orsay, France )
      • 16:15
        On the sensitivity of direct detection experiments to multi-component dark matter. 15m

        The Weakly Interacting Massive Particle or "WIMP" has been a widely studied solution to the dark matter problem. A plausible scenario is that DM is not made up of a single WIMP species, but that it has a multi-component nature. In this talk I give an overview of recently published work in which we studied direct detection signals in the presence of multi-component WIMP-like DM. I will give an overview of the "smoking gun" signature of two-component dark matter, as well as give a detailed explanation of the statistical methods used to forecast a signal in future generations of direct detection detectors. The two main avenues for forecasting that I will present involve a) discriminating between the one and two-component hypothesis and b) parameter reconstruction. I will also present an example of a minimal extension to the general model independent two-component phase space by introducing constraints from thermal freeze out.

        Speaker: Andre Scaffidi
      • 16:15
        Optimizing Provisioning of LCG Software Stacks with Kubernetes 15m

        The building, testing and deployment of coherent large software stacks is very challenging, in particular when they consist of the diverse set of packages required by the LHC experiments, the CERN Beams department and data analysis services such as SWAN. These software stacks comprise a large number of packages (Monte Carlo generators, machine learning tools, Python modules, HEP specific software) all available for several compilers, operating systems and hardware architectures. Along with several releases per year, development builds are provided each night to allow for quick updates and testing of development versions of packages such as ROOT, Geant4, etc. It also provides the possibility to test new compilers and new configurations.

        Timely provisioning of these development and release stacks requires a large amount of computing resources. A dedicated infrastructure, based on the Jenkins continuous integration system, has been developed to this purpose. Resources are taken from the CERN OpenStack cloud; Puppet configurations are used to control the environment on virtual machines, which are either used directly as resource nodes or as hosts for Docker containers. Containers are used more and more to optimize the usage of our resources and ensure a consistent build environment while providing quick access to new Linux flavours and specific configurations.

        In order to add build resources on demand more easily, we investigated the integration of a CERN provided Kubernetes cluster into the existing infrastructure.
        In this contribution we present the status of this prototype, focusing on the new challenges faced, such as the integration of these ephemeral build nodes into CERN’s IT infrastructure, job priority control, and debugging of job failures.

        Speaker: Gerardo Ganis (CERN)
      • 16:15
        Production Operations Management System (POMS) for Fermilab experiments 15m

        The Production Operations Management System (POMS) is a set of software tools which allows production teams and analysis groups across multiple Fermilab experiments to launch, modify and monitor large scale campaigns of related Monte Carlo or data processing jobs.
        POMS provides a web service interface that enables automated jobs submission on distributed resources according to customers’ requests and subsequent monitoring and recovery of failed submissions, debugging and record keeping.

        POMS interfaces with existing HEP data access, processing, movement and monitoring tools at Fermilab, including Jobsub, dCache, SAM, and FIFEmon, and in combination with them handles the creation and tracking of unique filenames for multistage campaigns.

        A flexible, interactive GUI interface exists as part of the system to visualize campaign stages, parallelization tracks and data flow. An important feature of POMS is a one-to-one connection between the GUI campaign visualizer and a text based representation of the complete campaign configuration. An extensible library of template campaign configurations is available. The templates are user modifiable and map cleanly to and from the GUI description of the campaign.

        Future releases of POMS will interface with Rucio for file movement.

        Speaker: Kenneth Richard Herner (Fermi National Accelerator Laboratory (US))
      • 16:15
        pyhf: a pure Python implementation of HistFactory with tensors and autograd 15m

        The HistFactory p.d.f. template [CERN-OPEN-2012-016] is per-se independent of its implementation in ROOT and it is useful to be able to run statistical analysis outside of the ROOT, RooFit, RooStats framework. pyhf is a pure-python implementation of that statistical model for multi-bin histogram-based analysis and its interval estimation is based on the asymptotic formulas of "Asymptotic formulae for likelihood-based tests of new physics" [arxiv:1007.1727]. pyhf supports modern computational graph libraries such as TensorFlow and PyTorch in order to make use of features such as autodifferentiation and GPU acceleration.

        Speaker: Matthew Feickert (Southern Methodist University (US))
      • 16:15
        Recent evolutions in the LTDP CDF project 15m

        For the latest years, the INFN-CNAF team has been working on the Long Term Data Preservation (LTDP) project for the CDF experiment, active at Fermilab from 1990 to 2011.
        The main aims of the project are to protect data of the CDF RUN-2 (4 PB) collected between 2001 and 2011 and already stored on CNAF tapes and to ensure the availability and the access to the analysis facility to those data over time.
        Lately, the CDF database, hosting information about CDF datasets such as their structure, file locations and metadata, has been imported from Fermilab to CNAF. The Sequential Access via Metadata (SAM) station data handling tool for CDF data management, that allows to manage data transfers and to retrieve information from the CDF database, has been properly installed and configured at CNAF. This is fundamental in a perspective of a complete decommissioning of CDF services on Fermilab side.
        An access system as been designed and tested to submit CDF analysis jobs, using CDF software distributed via CVMFS and requesting delivery of CDF files stored on CNAF tapes.
        Moreover, the availability and the correctness of all CDF data stored on CNAF tapes has been verified.
        This paper presents all these recent evolutions in detail, as well as the future plans for the LTDP project at CNAF.

        Speaker: luca dell'Agnello (INFN)
      • 16:15
        Reconstruction and analysis of ECal performance in the simulation of MpdRoot 15m

        As an important detector spectrum for the Nuclotron-based Ion Collider fAcility(NICA) accelerator complex at JINR, the MultiPurpose Detector(MPD) is proposed to investigate the hot and dense baryonic matter in heavy-ion collisions over a wide range of atomic masses, from Au+Au collisions at a centre-of-mass energy of $\sqrt{s_{nn}}=11GeV(for\ Au^{79+})$ to proton-proton collisions with $\sqrt{s_{pp}}=20GeV$.
        The performance of the MPD is simulated using MpdRoot based on the FairRoot framework. Considering the physics motivation of the MPD experiment, $\pi^0$ is a very important probe to give informations of the chiral symmetry restoration,flow signal and so on. So the performance of the electromagnetic calorimeter(ECAL) is reconstructed and studied, the ECAL is the main detector to measure the spatial position and energy of electrons and photon. The reconstruction of $\pi^0$ is carried on and the characteristics of the two-photon from $\pi^0$ decay is simulated and analyzed.

        Speaker: Dr Yan Huang (Tsinghua University)
      • 16:15
        SAGE – An Exascale Architecture based on Object Storage 15m

        The SAGE2 project is a collaboration between industry, data centres and research institutes demonstrating an exascale-ready system based on layered hierarchical storage and a novel object storage technology. The development of this system is based on a significant co-design exercise between all partners, with the research institutes having well established needs for exascale computing systems, both in terms of compute (exaFLOPS) and storage (exaBytes).

        In this paper we present an overview of the system design concepts and introduce some of the features of the object storage system which have the potential to make applications more efficient and simpler to use. The physical realisation of a prototype, hosted at Forschungszentrum Jülich is also detailed showing in detail the layered hierarchy

        We also present a case study of applications which have already been ported, covering issues uncovered and how these have been addressed in the codesign proves. Finally, we look at how this suite of applications are being extended into different regimes including machine learning to address a wider range of use cases and how interfaces are being developed to allow integration with other architectures

        Speaker: Andrew Davis (United Kingdom Atomic Energy Authority)
      • 16:15
        Setup and commissioning of a high-throughput analysis cluster 15m

        Current and future end-user analyses and workflows in High Energy Physics demand the processing of growing amounts of data. This plays a major role when looking at the demands in the context of the High-Luminosity-LHC. In order to keep the processing time and turn-around cycles as low as possible analysis clusters optimized with respect to these demands can be used. Since hyperconverged servers offer a good combination of compute power and local storage, they form the ideal basis for these clusters.
        In this contribution we report on the setup and commissioning of a dedicated analysis cluster setup at Karlsruhe Institute of Technology. This cluster was designed for use cases demanding high data-throughput. Based on hyperconverged servers this cluster offers 500 job slots and 1 PB of local storage. Combined with the 100 Gb network connection between the servers and a 200 Gb uplink to the Tier 1 storage, the cluster can sustain a data throughput of 1 PB per day.
        In addition the local storage provided by the hyperconverged worker nodes can be used as cache space. This allows employing of caching approaches on the cluster, thereby enabling a more efficient usage of the disk space. In previous contributions this concept has been shown to lead to an expected speedup of 2 to 4 compared to conventional setups.

        Speaker: Rene Caspart (KIT - Karlsruhe Institute of Technology (DE))
      • 16:15
        Signature of the chiral magnetic effect on the lattice. 15m

        Lattice quantum chromodynamics (QCD) has provided great insight into the nature of empty space, but quantum chromodynamics alone does not describe the vacuum in its entirety. Recent developments have introduced Quantum Electrodynamic (QED) effects directly into the generation of lattice gauge field configurations. Using lattice ensembles incorporating fully dynamical QCD and QED effects we are able to reveal for the first time evidence suggesting an interplay between these two forces. This is remarkable as gluons and photons, the force carriers of chromodynamics and electrodynamics respectively, do not interact directly and can only interact via the fully dynamical virtual quark-anti-quark pairs in the vacuum. By considering j.B and topological charge density q across the lattice we are able to quantitatively observe a correlation that may be associated with the chiral magnetic effect.

        Speaker: Mr Josh Charvetto (University of Adelaide)
      • 16:15
        Standalone containers with ATLAS offline software 15m

        This talk describes the deployment of ATLAS offline software in containers for use in production workflows such as simulation and reconstruction. For this purpose we are using Docker and Singularity, which are both lightweight virtualization technologies that can encapsulate software packages inside complete file systems. The deployment of offline releases via containers removes the interdependence between the runtime environment needed for job execution and the configuration of the computing nodes at the sites. Docker or Singularity would provide a uniform runtime environment for the grid, HPCs and variety of opportunistic resources. Additionally, releases may be supplemented with a detector’s conditions data, thus removing the need for network connectivity at computing nodes, which is normally quite restricted at HPCs.
        In preparation to achieve this goal, we have built Docker and Singularity images containing single full releases of ATLAS software for running event simulation jobs in runtime environments without network connection. These images have been successfully tested at the Theta supercomputer (ALCF) and at MareNostrum (BSC). Unlike similar parallel efforts to produce containers by packing all possible dependencies of every possible workflow into heavy images (~200GB), our approach is to include only what is needed for specific workflows and to manage dependencies efficiently via software package managers. This leads to more stable packaged releases where the dependencies are clear and the resulting images have more portable sizes (~16GB). In an effort to include a wider variety of workflows, we are deploying images that can be used in raw data reconstruction. This is particularly challenging due to the high database resource consumption during the access to the experiment’s conditions payload. We describe here a prototype pipeline in which images are provisioned only with the conditions payload necessary to satisfy the jobs’ requirements. This database-on-demand approach would keep images slim, portable and capable of supporting various workflows in a standalone fashion in environments with no network connectivity.

        Speaker: Alessandra Forti (University of Manchester (GB))
      • 16:15
        Status and future of the CMS Tracker DCS 15m

        Detector Control Systems (DCS) for modern High-Energy Physics (HEP) experiments are based on complex distributed (and often redundant) hardware and software implementing real-time operational procedures meant to ensuring that the detector is always in a "safe" state, while at the same time maximizing the live time of the detector during beam collisions. Display, archival and often analysis of the "environmental" data are also part of the tasks assigned to DCS systems. The CMS Tracker DCS is a resilient system that has been designed to safely operate the Silicon Strip and the Pixel detectors of the CMS detector. It has been built on top of an industrial Supervisory Control and Data Acquisition (SCADA) software product (WinCC OA) extended with the JCOP framework developed at CERN along with CMS and Tracker specific components. The Tracker control system is at present undergoing major component re-work which is critical to ensure a smooth Run 3 and a workable baseline for the Phase-2 Tracker upgrade. In this talk we will present an overview of the Tracker DCS, which has been operational for over 12 years, during which it has been continuously modified by several different hands, and of the major development challenges we are facing. Furthermore, we will discuss a generic way to build an automated test environment that ensures the proper functionality of different parts of the system. This includes defining test scenarios that cover individual components, integration and system testing using Kubernetes and Gitlab runners.

        Speaker: Wassef Karimeh (Université Saint-Joseph de Beyrouth (LB))
      • 16:15
        System testing CERN physics archival software using Docker and Kubernetes 15m

        CERN storage architecture is evolving to address run3 and run4 challenges. CTA and EOS integration requires parallel development of features in both software that needs to be synchronized and systematically tested on a specific distributed development infrastructure for each commit in the code base.

        CTA Continuous Integration development initially started as a place to run functional system tests against the freshly built software. But its importance grew over time to include all development, testing and deployment aspects.

        This presentation will focus on how we leverage Kubernetes and Docker to tackle various integration use cases.

        Speaker: Julien Leduc (CERN)
      • 16:15
        The ATLAS Hardware Track Trigger design towards first prototypes 15m

        In the High Luminosity LHC, planned to start with Run4 in 2026, the ATLAS experiment will be equipped with the Hardware Track Trigger (HTT) system, a dedicated hardware system able to reconstruct tracks in the silicon detectors with short latency. This HTT will be composed of about 700 ATCA boards, based on new technologies available on the market, like high speed links and powerful FPGAs, as well as custom-designed Associative Memories ASIC (AM), which are an evolution of those used extensively in previous experiments and in the ATLAS Fast Tracker (FTK).

        The HTT is designed to cope with the expected extreme high luminosity in the so called L0—only scenario, where HTT will operate at the L0 rate (1 MHz). It will provide good quality tracks to the software High-Level-Trigger (HLT), operating as coprocessor, reducing the HLT farm size by a factor of 10, by lightening the load of the software tracking.

        All ATLAS upgrade projects are designed also for an evolved, so-called "L0/L1" architecture, where part of HTT is used in a low-latency mode (L1Track), providing tracks in regions of ATLAS at a rate of up to 4MHz, with a latency of a few micro-seconds. This second phase poses very stringent requirements on the latency budget and to the dataflow rates.

        All the requirements and the specifications of this system have been assessed. The design of all the components has being reviewed and validated with preliminary simulation studies. After these validations are completed, the development of the first prototypes will start. In this paper we describe the status of the design review, showing challenges and assessed specifications, towards the preparation of the first slice tests with real prototypes.

        Speaker: Antonio Boveia (Ohio State University)
      • 16:15
        The evolution of the ALICE O2 monitoring system 15m

        ALICE (A Large Ion Collider Experiment) is currently ongoing a major upgrade of the detector, read-out and computing system for LHC Run 3. A new facility called O2 (Online-Offline) will perform data acquisition and event processing.
        To efficiently operate the experiment and the O2 facility a new observability system has been developed. It will provide a complete overview of the overall health, detect performance degradation and component failures by collecting, processing, storing and visualizing values from hardware and software sensors and probes. The core of the system is based on Apache Big Data tools, InfluxData time-series components and Grafana.
        Recent major changes, as adapting Apache Kafka as a metric collector and processor, lead to a more generic system design that, in addition to monitoring, is capable of dealing with logs and request tracing data.
        This paper describes the system design and its evolution, reasoning behind adapting new components, performance and latency measurements, challenges with the scaling, stability tests and the automatic deployment process.

        Speaker: Adam Wegrzynek (CERN)
      • 16:15
        The eXtreme-DataCloud project - solutions for data management services in distributed e-infrastructures 15m

        The eXtreme DataCloud (XDC) project is aimed at developing data management services capable to cope with very large data resources allowing the future e-infrastructures to address the needs of the next generation extreme scale scientific experiments. Started in November 2017, XDC is combining the expertise of 8 large European research organisations, the project aims at developing scalable technologies for federating storage resources and managing data in highly distributed computing environments.
        The project is use case driven with a multidisciplinary approach, addressing requirements from research communities belonging to a wide range of scientific domains: Life Science, Biodiversity, Clinical Research, Astrophysics, High Energy Physics and Photon Science, that represent an indicator in terms of data management needs in Europe and worldwide.
        The use cases proposed by the different user communities are addressed integrating different data management services ready to manage an increasing volume of data. Different scalability and performance tests have been defined to show that the XDC services can be harmonized in different contexts and complex frameworks like the European Open Science Cloud.
        The use cases have been used to measure the success of the project and to prove that the developments fulfill the defined needs and satisfy the final users.
        The present contribution describes the results carried out from the adoption of the XDC solutions and provides a complete overview of the project achievements.

        Speaker: Patrick Fuhrmann
      • 16:15
        The GridKa Tape Storage: various performance test results and current improvements 15m

        Data growth over several years within HEP experiments requires a wider use of storage systems for WLCG Tiered Centers. It also increases the complexity of storage systems, which includes the expansion of hardware components and thereby complicates existing software products more. To cope with such systems is a non-trivial task and requires highly qualified specialists.

        Storing petabytes of data on tape storage is a still the most cost-effective way. Year after year, the use of a tape storage increases, consequently a detailed study of its optimal use and verification of performance is a key aspect for such a system. It includes several factors, such as performing various performance tests, identifying and eliminating bottlenecks, properly adjusting and improving the current GridKa setup, etc.

        At present, GridKa uses dCache as the storage system in frontend and TSM as the tape storage backend. dCache provides a plugin interface for exchanging data between dcache and tape.

        TSS is a TSM-based client developed by the GridKa team. TSS has been in production for over 10 years. The interaction between the GridKa dCache instance and TSM is accomplished using additional scripts that can be further optimized to improve the overall performance of the tape storage.

        This contribution provides detailed information on the results of various performance tests performed on the GridKa tape and significant improvements of our tape storage performance.

        Speaker: Haykuhi Musheghyan (Georg August Universitaet Goettingen (DE))
      • 16:15
        The upgrade and re-validation of the Compact Muon Solenoid Electromagnetic Calorimeter Control and Safety Systems during the second Long Shutdown of the Large Hadron Collider at CERN 15m

        The Electromagnetic Calorimeter (ECAL) is one of the sub-detectors of the Compact Muon Solenoid (CMS), a general-purpose particle detector at the CERN Large Hadron Collider (LHC). The CMS ECAL Detector Control System (DCS) and the CMS ECAL Safety System (ESS) have supported the detector operations and ensured the detector's integrity since the CMS commissioning phase, more than 10 years ago. Over this long period, several changes to both systems were necessary to correct issues, extend functionality and keep them in-line with current hardware technologies and the evolution of software platforms. Due to the constraints imposed on significant changes to a running system, major hardware and software upgrades were therefore deferred to the second LHC Long Shutdown (LS2). This paper presents the architectures of the CMS ECAL control and safety systems, discusses the ongoing and planned upgrades, details implementation processes and validation methods and highlights the expectations for the post-LS2 systems.

        Speaker: Mr Raul Jimenez Estupinan (ETH Zurich (CH))
      • 16:15
        The Virtual Geometry Model 15m

        The Virtual Geometry Model (VGM) is a geometry conversion tool, currently providing conversion between Geant4 and ROOT TGeo geometry models. Its design allows the inclusion of another geometry model by implementing a single sub-module instead of writing bilateral converters for all already supported models.
        The VGM was last presented at CHEP in 2008 and since then it has been under continuous maintenance and development, following the evolutions of the supported geometry models and adapting to different use cases. Being integrated in Geant4 VMC for the support of TGeo geometry definition with Geant4 native geometry navigation or the support of Geant4 geometry definition with Geant3 TGeo navigation, it is used in large experimental frameworks, such as FairRoot or ALICE O2.
        In this presentation, we will give an update on the tool architecture, implementation and supported features, user examples, testing and documentation. We will also present the tool build system, distribution and releases policy. Finally, we will discuss the possibilities of using the tool for verification of user geometries.

        Speaker: Ivana HRIVNACOVA (Institut de Physique Nucléaire (IPNO), Université Paris-Sud, CNRS-IN2P3, Orsay, France )
      • 16:15
        Tile-in-One: An integrated system for data quality and conditions assessment for the ATLAS Tile Calorimeter 15m

        The Tile Calorimeter (TileCal) is a crucial part of the ATLAS detector which jointly with other calorimeters reconstructs hadrons, jets, tau-particles, missing transverse energy and assists in muon identification. It is constructed of alternating iron absorber layers and active scintillating tiles and covers region |eta| < 1.7. The TileCal is regularly monitored by several different systems, which were developed mainly during the commissioning of the detector in order to meet distinct collaboration's requirements. Any problems are reported and immediately investigated, which results in data quality efficiency very close to 100% achieved in last several years. Although the TileCal tools are maintained, the underlying technologies are becoming gradually outdated.

        The Tile-in-One strives to integrate all data quality and conditions assessment TileCal tools into one common system. This system is implemented as a web application with main machine being the gateway for so-called plugins. The plugin is a standalone small web application hosted on a single virtual machine. The plugins are separated into virtual machines due to the requirement for different data sources and to avoid interference in order to increase stability of the platform. The main server is responsible for authentication and authorization of the users as well as the management of the plugins. Currently the platform consists of 13 plugins in various stages of development. The implementation details of the Tile-in-One web system and as well as selected plugins will be presented.

        Speaker: Iouri Smirnov (Northern Illinois University (US))
      • 16:15
        Token-based authorization in the StoRM WebDAV service 15m

        Support for token-based authentication and authorization has emerged in recent years as a key requirement for storage elements powering WLCG data centers. Authorization tokens represent a flexible and viable alternative to other credential delegation schemes (e.g. proxy certificates) and authorization mechanisms (VOMS) historically used in WLCG, as documented in more detail in other submitted contributions to this conference.

        In this contribution, we describe the work done to enable token-based authentication and authorization in the StoRM WebDAV service, describing and highlighting the differences between support for external OpenID connect providers, group-based and capability-based authorization schemes, and locally-issued authorization tokens.

        We also discuss how StoRM WebDAV token-based authorization is being exploited in several contexts, from WLCG DOMA activities to other scientific experiments hosted at the INFN Tier-1 data center.

        Speaker: Andrea Ceccanti (Universita e INFN, Bologna (IT))
      • 16:15
        Using ML to Speed Up New and Upgrade Detector Studies 15m

        Designing new experiments, as well as upgrade of ongoing experiments, is a continuous process in experimental high energy physics. Frontier R&Ds are used to squeeze the maximum physics performance using cutting edge detector technologies.

        The evaluating of physics performance for particular configuration includes sketching this configuration in Geant, simulating typical signals and backgrounds, applying reasonable reconstruction procedures, combining results into final quality metrics. Since the best solution is always a trade-off between different kinds of limitations, a quick turn over is necessary to evaluate physics performance for different technical solutions in different configurations.

        Two typical problems which slow down evaluating physics performance for different detector technologies and configurations are: describing Geant geometry together with signal processing chain for an adequate description of the detector response and developing adequate reconstruction algorithm for physics reconstruction of detector response under study. Both problems may be addressed using modern ML approaches. In addition to this, the whole procedure can be viewed as a black-box optimisation, which gives access to numerous available methods.

        In our presentation, we discuss the way advanced machine learning techniques allow to speed up the detector development and optimization cycle with an emphasis on the project of the calorimeter upgrade for the LHCb detector.

        Speaker: Fedor Ratnikov (Yandex School of Data Analysis (RU))
      • 16:15
        Visualising the Emergent Nonperturbative Structure of QCD 15m

        The gluon field configurations that form the foundation of every lattice QCD calculation contain a rich diversity of emergent nonperturbative phenomena. Visualisation of these phenomena creates an intuitive understanding of their structure and dynamics. This presentation will illustrate recent advances in observing the chromo-electromagnetic vector fields, their energy and topological charge densities, and the manner in which vortices in the gluon fields percolate space-time.

        Speaker: Derek Leinweber (CSSM, University of Adelaide)
    • 16:30 18:00
      Track 1 – Online and Real-time Computing: Detectors, performance, and analysis Riverbank R5

      Riverbank R5

      Adelaide Convention Centre

      Convener: Yu Nakahama Higuchi (Nagoya University (JP))
      • 16:30
        Strategies for detecting long-lived particles at LHC experiments 15m

        The detection of long-lived particles (LLPs) in high energy experiments are key for both the study of the Standard Model (SM) of particle physics and to search for new physics beyond it.
        Many interesting decay modes involve strange particles with large lifetimes such as Ks or L0s. Exotic LLP are also predicted in many new theoretical models. The selection and reconstruction of LLPs produced in proton-proton collisions at the Large Hadron Collider (LHC) at CERN is a challenge. These particles can decay far from the primary interaction vertex and are hard to select by the trigger systems of the experiments and difficult to isolate from the SM backgrounds. In this talk several strategies followed by ATLAS, CMS and LHCb experiments are reviewed. The importance of new hardware architectures in the first stage of the high level trigger (HLT), making them more performant in terms of flexibility, speed and efficiency, is outlined. In particular, the use of accelerators for higher pile-up conditions during the HL-LHC era is presented.

        Speaker: Mr Brij Kishor Jashal (Tata Inst. of Fundamental Research (IN))
      • 16:45
        Recording and reconstructing 10 billion unbiased B meson decays in CMS to probe lepton flavour universality 15m

        Measurements involving rare B meson decays by the LHCb and Belle Collaborations have revealed a number of anomalous results. Collectively, these anomalies are generating significant interest in the community, as they may be interpreted as a first sign of new physics in the lepton flavour sector. In 2018, the CMS experiment recorded an unprecedented data set containing the unbiased decays of 10 billion B mesons. These data allow attempts to measure the R_K, R_K*, and R_phi observables, which are considered to be theoretically robust and sensitive to a violation of lepton flavour universality. The accumulation, processing, and validation of this unprecedented data set was delivered with negligible impact on the core physics programme of CMS. Events were recorded with trigger rates in excess of 50 and 5 kHz at L1 and HLT, respectively, and a high average throughput of 2 GB/s. New algorithms have been developed to reconstruct and identify electrons with high efficiency at very low transverse momenta, as low as 0.5 GeV. These algorithms were validated using a pilot reconstruction campaign of O(10^9) events. The entire data set is currently being reconstructed and stored in the MINIAOD data format, along with a signal-enriched skim of events in the RAW data format that will aid the commissioning of the analysis techniques. We will review these activities, as well as future plans for Run 3.

        Speaker: Robert John Bainbridge (Imperial College (GB))
      • 17:00
        The CMS electromagnetic calorimeter workflow 15m

        The CMS experiment at the LHC features the largest crystal electromagnetic calorimeter (ECAL) ever built. It consists of about 75000 scintillating lead tungstate crystals. The ECAL crystal energy response is fundamental for both triggering purposes and offline analysis. Due to the challenging LHC radiation environment, the response of both crystals and photodetectors to particles evolves with time. Therefore continuous monitoring and correction of the ageing effects plays a key role in ensuring stability of trigger efficiency and rate, as well as optimal offline physics performance. Fast, reliable and efficient workflows are set up to have the first set of corrections computed within 48 hours of data-taking, making use of dedicated data streams and processing. Such corrections, stored in relational databases (Oracle), are then accessed during the prompt offline reconstruction of CMS data. Twice a week, the calibrations used at the trigger level (hardware and software) are also updated in the database and accessed during the data-taking. Dedicated workflows for the prompt validation of the conditions are also available, together with a monitoring of the data quality. In this presentation, the design of the CMS ECAL data handling, processing, and validation is reviewed, and results from the main workflows are discussed.

        Speaker: Chiara Ilaria Rovelli (Sapienza Universita e INFN, Roma I (IT))
      • 17:15
        Time measurement with the SND electromagnetic calorimeter 15m

        The SND is a non-magnetic detector deployed at the VEPP-2000 e+e- collider (BINP, Novosibirsk) for hadronic cross-section measurements in the center of mass energy region below 2 GeV. The important part of the detector is a three-layer hodoscopic electromagnetic calorimeter (EMC) based on NaI(Tl) counters. Until the recent EMC spectrometric channel upgrade, only the energy deposition measurement in counters was possible. A new EMC signal shaping and digitizing electronics based on FADC allowed us to obtain also the signal arrival time. The new electronics and supporting software, including digitized signal processing algorithms, are now used for data taking in the ongoing experiment. We discuss the amplitude and time extraction algorithms, the new system performance on experimental events and how this time measurement can be applied to physics analysis.

        Speaker: Natalya Melnikova (Budker Institute of Nuclear Physics (RU))
      • 17:30
        Track reconstruction with PANDA at FAIR 15m

        The upcoming PANDA experiment is one of the major pillars of the future FAIR accelerator facility in Darmstadt, Germany. With its multipurpose detector and an antiproton beam with a momentum of up to 15 GeV/c, PANDA will be able to test QCD in the intermediate energy regime and shed light on important questions such as: Why is there a matter-antimatter asymmetry in the Universe?

        Achieving its physics goals requires PANDA to perform exclusive measurements of the many desired reaction channels as well as being on the intensity and precision frontiers. The foreseen average event rate will be as high as 20 MHz. At the same time, due to the similarity of signal and background processes, the experiment will need to employ a purely software-based event selection. To facilitate this, tracks and events have to be fully reconstructed in real time, using the available information from all detector components. These conditions prove to be very challenging for the established reconstruction algorithms, and substantial adaptations are mandatory. Fully utilising modern, heterogeneous computing environments, including CPUs, GPUs, and FPGAs will be essential.

        This presentation will focus on methods that are developed for PANDA's Straw Tube Tracker. These include an adaptation of a cellular automaton to operate on free-streaming data, as well as an algorithm for longitudinal momentum reconstruction with inclined straw tubes.

        Speaker: Michael Papenbrock (Uppsala University)
      • 17:45
        The ATLAS Muon, Electron and Photon Trigger Performance 15m

        Events containing muons, electrons or photons in the final state are an important signature for many analyses being carried out at the Large Hadron Collider (LHC), including both standard model measurements and searches for new physics. To be able to study such events, it is required to have an efficient and well-understood trigger system. The ATLAS trigger consists of a hardware based system (Level 1), as well as software based reconstruction (High Level Trigger). To cope with ever-increasing luminosity and more challenging pile-up conditions at the LHC, several improvements have been implemented to keep the trigger rate low, while still maintaining high efficiency. We will present an overview of how we trigger on muons, electrons and photons, recent improvements, the performance of these triggers in Run-2 data and the improvements planned for Run-3.

        Speaker: Batool Safarzadeh Samani (University of Sussex (GB))
    • 16:30 18:00
      Track 2 – Offline Computing: Cross-experiment frameworks & foundations Riverbank R6

      Riverbank R6

      Adelaide Convention Centre

      Convener: Chris Pinkenburg
      • 16:30
        DD4hep: a community driven detector description tool for HEP 15m

        Detector description is an essential component in simulation, reconstruction and analysis of data resulting from particle collisions in high energy physics experiments and for the detector development studies for future experiments. Current detector description implementations of running experiments are mostly specific implementations. DD4hep is an open source toolkit created in 2012 to serve as a generic detector description solution. The main motivation behind DD4hep is to provide the community with an integrated solution for all these stages and address detector description in a broad sense, including the geometry and the materials used in the device, and additional parameters describing e.g. the detection techniques, constants required for alignment and calibration, description of the readout structures and conditions data. In this presentation, we will give an overview of the project and discuss recent developments in DD4hep as well as showcase adaptions of the framework by LHC and upcoming accelerator projects together with the roadmap of future developments. We will describe the DDG4 component of DD4hep, which is a powerful tool that converts arbitrary DD4hep detector geometries to Geant4 and gives access to all Geant4 action stages, including an overview of its comprehensive plugin suite that includes handling of different IO formats, Monte Carlo truth linking and a large set of segmentation and sensitive detector classes, allowing the simulation of a wide variety of detector technologies. We will further describe DDCond and DDAlign, which expose a mechanism to manage multiple versions of detector conditions data simultaneously and efficiently.

        Speaker: Frank-Dieter Gaede (Deutsches Elektronen-Synchrotron (DE))
      • 16:45
        CMS Experience with Adoption of the Community-supported DD4hep Toolkit 15m

        DD4hep is an open-source software toolkit that provides comprehensive and complete generic detector descriptions for high energy physics (HEP) detectors. The Compact Muon Solenoid collaboration (CMS) has recently evaluated and adopted DD4hep to replace its custom detector description software. CMS has demanding software requirements as a very large, long-running experiment that must support legacy geometries and study many possible upgraded detector designs of a constantly evolving detector that will be taking data for many years to come. CMS has chosen DD4hep since it is a high-quality, community-supported solution that will benefit from continuing modernization and maintenance. This presentation will discuss the issues of DD4hep adoption, the advantages and disadvantages of the various design choices, performance results, and the integration of the plugin systems from CMS and Gaudi, another open-source software framework. Recommendations about DD4hep based upon the CMS use cases will also be presented.

        Speaker: Dr Carl Vuosalo (University of Wisconsin Madison (US))
      • 17:00
        Evolution of the ROOT Tree I/O 15m

        The ROOT TTree data format encodes hundreds of petabytes of High Energy and Nuclear Physics events. Its columnar layout drives rapid analyses, as only those parts (branches) that are really used in a given analysis need to be read from storage. Its unique feature is the seamless C++ integration, which allows users to directly store their event classes without explicitly defining data schemas. In this contribution, we present the status and plans of the future ROOT 7 event I/O. Along with the ROOT 7 interface modernization, we aim for robust, where possible compile-time safe C++ interfaces to read and write event data. On the performance side, we show first benchmarks using ROOT's new experimental I/O subsystem that combines the best of TTrees with recent advances in columnar data formats. A core ingredient is a strong separation of the high-level logical data layout (C++ classes) from the low-level physical data layout (storage backed nested vectors of simple types). On the high level, we present new asynchronous and thread-friendly interfaces to support parallel reading and writing. On the low-level, we show how an optimized physical data layout speeds up serialization and deserialization and facilitates fast vectorized and bulk operations. This lets ROOT I/O run optimally on the upcoming ultra-fast NVRAM storage devices, as well as file-less storage systems such as object stores.

        Speaker: Jakob Blomer (CERN)
      • 17:15
        Automatic Differentiation in ROOT 15m

        In mathematics and computer algebra, automatic differentiation (AD) is a set of techniques to evaluate the derivative of a function specified by a computer program. AD exploits the fact that every computer program, no matter how complicated, executes a sequence of elementary arithmetic operations (addition, subtraction, multiplication, division, etc.) and elementary functions (exp, log, sin, cos, etc.) including control flow statements. AD takes source code of a function as input and produces source code of the derived function. By applying the chain rule repeatedly to these operations, derivatives of arbitrary order can be computed automatically, accurately to working precision, and using at most a small constant factor more arithmetic operations than the original program.

        AD is an alternative technique to symbolic and numerical differentiation. These classical methods run into problems: symbolic differentiation leads to inefficient code (unless done carefully) and faces the difficulty of converting a computer program into a single expression, while numerical differentiation can introduce round-off errors in the discretization process and cancellation. Both classical methods have problems with calculating higher derivatives, where the complexity and errors increase. Finally, both classical methods are slow at computing the partial derivatives of a function with respect to many inputs, as is needed for gradient-based optimization algorithms. AD solves all of these problems, at the expense of introducing more software dependencies.

        Our talk presents AD techniques available in ROOT, supported by Cling, to produce derivatives of arbitrary C/C++ functions through implementing source code transformation and employing the chain rule of differential calculus in both forward mode and reverse mode. We explain it's current integration for gradient computation in TFormula. We demonstrate the correctness and performance improvements in ROOT's fitting algorithms.

        Speaker: Oksana Shadura (University of Nebraska Lincoln (US))
      • 17:30
        Review of High-Quality Pseudo Random Number Generators 15m

        Pseudo random number generation (PRNG) play an important role in many areas of computational science. Highest random properties, exact reproducibility and CPU efficiency are important requirements for using them in the most demanding Monte Carlo calculations.
        We are reviewing here the highest quality PRNG available, such as those based on the Kolmogorov-Anosov theory of mixing in classical mechanical systems. The theory guarantees under certain conditions that points on the trajectories of these systems can be used to produce random number sequences of exceptional quality.
        We outline this theory of mixing and establish criteria for deciding which RNG’s are sufficiently good approximations to the ideal mathematical systems that guarantee highest quality.
        We will present these type of generator and among those, RANLUX, its recent variant RANLUX++ and MIXMAX, a new matrix generator recently proposed. We will show how some of the proposed versions of MIXMAX can be modified easily to meet our criteria for highest quality.
        We will present as well results of the most stringent empirical tests applied to these generators and commonly used ones and report on their CPU run time performances.

        Speaker: Lorenzo Moneta (CERN)
      • 17:45
        Using multiple engines in the Virtual Monte Carlo package 15m

        The Virtual Monte Carlo (VMC) package together with its concrete implementations provides a unified interface to different detector simulation transport engines such as GEANT3 or GEANT4. However, so far the simulation of one event was restricted to the usage of one chosen engine.

        We introduce here the possibility to mix multiple engines within the simulation of one event. Depending on user conditions the simulation is then effectively split among the chosen engines. Among others, these conditions can depend on phase space, geometry, particle type or an arbitrary combination. This development hence offers the possibility to choose multiple engines for an event simulation profiting from each of their advantages or specific capabilities under different conditions.

        Furthermore, it allows for the implementation of fast simulation kernels on VMC level which can then easily be dispatched to during simulation. This also makes stand-alone fast simulation possible since such a fast simulation kernel in essence acts as a fully functional engine.

        This development opens many new opportunities for the usage of the VMC package and its functionality. It allows for extended studies of detector simulation engines, their interplay and, especially concerning customized user implementations of fast simulations, it provides vital new flexibility.

        Speaker: Benedikt Volkel (Ruprecht Karls Universitaet Heidelberg (DE), CERN)
    • 16:30 18:00
      Track 3 – Middleware and Distributed Computing: Security & Collaboration Riverbank R3

      Riverbank R3

      Adelaide Convention Centre

      Convener: Catherine Biscarat (L2I Toulouse, IN2P3/CNRS (FR))
      • 16:30
        WLCG Authorisation; from X.509 to Tokens 15m

        The WLCG Authorisation Working Group formed in July 2017 with the objective to understand and meet the needs of a future-looking Authentication and Authorisation Infrastructure (AAI) for WLCG experiments. Much has changed since the early 2000s when X.509 certificates presented the most suitable choice for authorisation within the grid; progress in token based authorisation and identity federation has provided an interesting alternative with notable advantages in usability and compatibility with external (commercial) partners. The need for interoperability in this new model is paramount as infrastructures and research communities become increasingly interdependent. Over the past two years, the Working Group has made significant steps towards identifying a system to meet the technical needs highlighted by the community during staged requirements gathering activities. Enhancement work has been possible thanks to externally funded projects, allowing existing AAI solutions to be adapted to our needs. A cornerstone of the infrastructure is the reliance on a common token schema in line with evolving standards and best practices, allowing for maximum compatibility and easy cooperation with peer infrastructures and services. We will present the work of the group and an analysis of the anticipated changes in authorisation model by moving from X.509 to token based authorisation. A concrete example of token integration in Rucio, FTS and storage (dCache and StoRM) will be discussed.

        Speaker: Andrea Ceccanti (Universita e INFN, Bologna (IT))
      • 16:45
        Harnessing the power of threat intelligence for WLCG cybersecurity 15m

        The information security threats currently faced by WLCG sites are both sophisticated and highly profitable for the actors involved. Evidence suggests that targeted organisations take on average more than six months to detect a cyber attack, with more sophisticated attacks being more likely to pass undetected.

        An important way to mount an appropriate response is through the use of a Security Operations Centre (SOC). A SOC can provide detailed traceability information along with the capability to quickly detect malicious activity. The core building blocks of such a SOC are an Intrusion Detection System and a threat intelligence component, required to identify potential cybersecurity threats as part of a trusted community. The WLCG Security Operations Centre Working Group has produced a reference design for a minimally viable Security Operations Centre, applicable at a range of WLCG sites.

        While the fundamental technologies required for this approach are relatively well understood, with much of the technical capability to provide WLCG sites with threat intelligence already in place, an important factor in the sharing of threat intelligence is the formation of appropriate trust groups.

        We present the approach of the working group to facilitating the collaboration necessary to form these groups, including both technological and social aspects, along with our most recent results. We emphasise the importance of collaboration not only between WLCG sites, but also between grid and campus teams. This type of broad collaboration is essential given the nature of threats faced by the WLCG, which can often be a result of compromised campus resources.

        Speaker: David Crooks (Science and Technology Facilities Council STFC (GB))
      • 17:00
        A New Authorization System for IceCube Applications 15m

        As part of a modernization effort at IceCube, a new unified authorization system has been developed to allow access to multiple applications with a single credential. Based on SciTokens and JWT, it allows for the delegation of specific accesses to cluster jobs or third party applications on behalf of the user. Designed with security in mind, it includes short expiration times on access tokens, refresh tokens, and a revocation option.

        Speaker: David Schultz (University of Wisconsin-Madison)
      • 17:15
        Computer security in 2019 15m

        In this talk, the speaker will present the computer security risk landscape as faced by academia and research organisations; look into various motivations behind attacks; and explore how these threats can be addressed. This will be followed by details of several types of vulnerabilities and incidents recently affecting HEP community, and lessons learnt. The talk will conclude with a outlook into possible future trends, and ways of responding to them.

        Speaker: Sebastian Lopienski (CERN)
      • 17:30
        Building an IRIS trust framework 15m

        IRIS is the co-ordinating body of a UK science eInfrastructure and is a collaboration between UKRI-STFC, its resource providers and representatives from the science activities themselves. We document the progress of an ongoing project to build a security policy trust framework suitable for use across the IRIS community.

        The EU H2020-funded AARC projects addressed the challenges involved in integrating identity services across different infrastructures, thereby allowing research communities to securely share data and resources. The result of this work hinged around the AARC Blueprint Architecture allowing federations of services and identity providers to connect via one or more proxies. In addition to AARC technical architecture documents and guidelines, a policy team created a set of template policies published as the AARC Policy Development Kit (PDK), which following the completion of the AARC projects will find a long term home under the SCI working group of the WISE community. Derived from existing practice, the PDK aims to assist in efficiently bootstrapping Research Infrastructures in the operation of an authentication and authorisation infrastructure in line with the AARC Blueprint Architecture, making them accessible to researchers in an easy and secure fashion.

        We document the experience gained by the IRIS community in adopting component policies of the PDK to form a policy foundation for resource sharing, access and trust. The lack of such an established trust framework hindered the early growth of IRIS activities and the promise of a ‘ready-made’ framework of templates, based on current best practice recommendations for federated environments, available for use made the PDK an obvious place to start. Starting with an examination of stakeholder requirements and discussion of how to map these to the PDK templates, we describe the current status of the process to create a sub-set of policies to form a trust framework for a national infrastructure collaboration.

        Speaker: David Crooks (Science and Technology Facilities Council STFC (GB))
      • 17:45
        China-EU scientific cooperations on JUNO distributed computing 15m

        The Jiangmen Underground Neutrino Observatory (JUNO) is an underground 20 kton liquid scintillator detector being built in the south of China and expected to start data taking in late 2021. The JUNO physics program is focused on exploring neutrino properties, by means of electron anti-neutrinos emitted from two nuclear power complexes at a baseline of about 53km. Targeting an unprecedented relative energy resolution of 3% at 1 MeV, JUNO will be able to study neutrino oscillation phenomena and determine neutrino mass ordering with a statistical significance of 3-4 sigma within six years running time.

        These physics challenges are addressed by a large Collaboration localized in three continents. In this context, key to the success of JUNO will be the realization of a distributed computing infrastructure, which will satisfy its predicted computing needs.

        The development of the computing infrastructure is performed jointly by the Institute for High Energy Physics (IHEP) (part of Chinese Academy of Sciences (CAS)), and a number of Italian, French and Russian data centers, already part of WLCG.

        Upon its establishment, JUNO is expected to deliver not less than 2 PB of data per year, to be stored in the above mentioned data centers throughout China and Europe. Data analysis activities will be also carried out in cooperation, according to a coordinated joint effort.

        This contribution is meant to report on China-EU cooperation to design and build together the JUNO computing infrastructure and to describe its main characteristics and requirements.

        Speaker: Giuseppe Andronico (Universita e INFN, Catania (IT))
    • 16:30 18:00
      Track 4 – Data Organisation, Management and Access: workflows and infrastructures Riverbank R8

      Riverbank R8

      Adelaide Convention Centre

      Convener: Xavier Espinal (CERN)
      • 16:45
        Transitioning CMS to Rucio Data Management 15m

        Following a thorough review in 2018, the CMS experiment at the CERN LHC decided to adopt Rucio as its new data management system. Rucio is emerging as a community software project and will replace an aging CMS-only system before the start-up of LHC Run 3 in 2021. Rucio was chosen after an evaluation determined that Rucio could meet the technical and scale needs of CMS. The data management system for CMS needs to manage the current data sample (approximately 200 PB of data with 1 PB of transfers per day) and have a development path suitable for LHC Run 4 (2026) data rates, which are expected to be 50 times larger.

        This contribution will detail the ongoing CMS adoption process as we replace our legacy system with Rucio, focusing on the challenges of integrating Rucio into an established set of experimental tools and procedures. This will include the migration of metadata, the construction of interfaces to the rest of the CMS computing tools, scale tests, operations, monitoring, and the plan to gradually turn over primary responsibility for the management of data to Rucio. A description of CMS user data management with Rucio will also be provided.

        Speaker: Eric Vaandering (Fermi National Accelerator Lab. (US))
      • 17:00
        The Dynafed data federator as grid site storage element 15m

        The Dynafed data federator is designed to present a dynamic and unified view of a distributed file repository. We describe our use of Dynafed to construct a production-ready WLCG storage element (SE) using existing Grid storage endpoints as well as object storage. In particular, Dynafed is used as the primary SE for the Canadian distributed computing cloud systems. Specifically, we have been using a Dynafed-based SE in production for reading input files of Belle-II jobs. We have run up to 6000 Belle-II jobs simultaneously on distributed cloud resources, requiring the transfer of approximately 60 TB of input data per day. Similarly, we have been using a Dynafed-based SE in pre-production testing for the ATLAS experiment. We will describe the configuration of Dynafed to make it suitable as a WLCG SE. In particular, we will highlight the improvements within Dynafed that make it possible to do checksum based file verification and 3rd-party file copy via WebDAV. We will also report on a new monitoring system and an automated system that collects storage information from all endpoints. The monitoring is accessible via a web browser and also on the command line. It includes, for example, information of who requested files, which endpoints served the files, how often each file was requested from an endpoint, as well as endpoint storage information collected by the system we developed. The storage information system collects information about each storage endpoint which is then used by Dynafed to identify endpoints with sufficient storage capacity. The presentation will discuss the configuration and successful operation of Dynafed within the context of a distributed compute cloud infrastructure.

        Speaker: Frank Berghaus (University of Victoria (CA))
      • 17:15
        Data migration strategy based on file heat prediction with deep learning methods 15m

        As a data-intensive computing application, high-energy physics requires storage and computing for large amounts of data at the PB level. Performance demands and data access imbalances in mass storage systems are increasing. Specifically, on one hand, traditional cheap disk storage systems have been unable to handle high IOPS demand services. On the other hand, a survey found that only a very small number of files have been active in storage for a period of time. Tiered storage architectures, such as tape, disk or solid state drives were used to reduce hardware purchase costs and power consumption.

        As the amount of stored data grows, tiered storage requires data management software to migrate less active data to lower cost storage devices. Thus an automated data migration strategy is needed. At present, automatic data migration strategies such as LRU, CLOCK, 2Q, GDSF, LFUDA, FIFO, etc., are usually based on files’ recent access mode(such as file access frequency, etc.), are mainly used to resolve data migration between memory and disk. They need to run in the operating system kernel, so the rules are relatively simple. For file access mode does not take file life cycle trend into account, some regularly accessed files are often not predicted accurately. In addition, file history access records are not considered.

        Data access requests are not completely random. They are driven by the behavior of users or programs. There must be association between different files that are accessed consecutively. This paper proposes a method of file access heat prediction. Data heat trend is used as the basis for migration to a relatively low-cost storage device. Due to the limitations of traditional models, it is difficult to achieve good results in predicting at such nonlinear scenes. This paper attempts to use the deep learning algorithm model to predict the evolution trend of data access heat. This paper discussed the implementation of some initial parts of the system, in particular the trace collector and the LSTM model. Then some preliminary experiments are conducted with these parts.

        Speakers: Shiyuan Fu, Shiyuan Fu
      • 17:30
        ESCAPE prototypes a Data Infrastructure for Open Science 15m

        The European-funded ESCAPE project will prototype a shared solution to computing challenges in the context of the European Open Science Cloud. It targets Astronomy and Particle Physics facilities and research infrastructures and focuses on developing solutions for handling Exabyte scale datasets.

        The DIOS work package aims at delivering a Data Infrastructure for Open Science. Such an infrastructure would be a non HEP specific implementation of the data lake concept elaborated in the HSF Community White Paper and endorsed in the WLCG Strategy Document for HL-LHC.

        The science projects in ESCAPE are in different phases of evolution. While HL-LHC can leverage 15 years of experience of distributed computing in WLCG, other sciences are building now their computing models. This contribution describes the architecture of a shared ecosystem of services fulfilling the needs in terms of data organisation, management and access for the ESCAPE community. The backbone of such a data lake will consist of several storage services operated by the partner institutes and connected through reliable networks. Data management and organisation will be orchestrated through Rucio. A layer of caching and latency hiding services, supporting various access protocols will serve the data to heterogeneous facilities, from conventional Grid sites to HPC centres and Cloud providers. The authentication and authorisation system will be based on tokens.

        For the success of the project, DIOS will integrate open source solutions which demonstrated reliability and scalability as at the multi petabyte scale. Such services will be configured, deployed and complemented to cover the use cases of the ESCAPE sciences which will be further developed during the project.

        Speaker: Simone Campana (CERN)
      • 17:45
        SkyhookDM: Mapping Scientific Datasets to Programmable Storage 15m

        Access libraries such as ROOT and HDF5 allow users to interact with datasets using high level abstractions, like coordinate systems and associated slicing operations. Unfortunately, the implementations of access libraries are based on outdated assumptions about storage systems interfaces and are generally unable to fully benefit from modern fast storage devices. For example, access libraries often implement buffering and data layout that assume that large, single-threaded sequential access patterns are causing less overall latency than small parallel random access: while this is true for spinning media, it is not true for flash media. The situation is getting worse with rapidly evolving storage devices such as non-volatile memory and ever larger datasets. Our Skyhook Dataset Mapping project explores ​distributed dataset mapping ​infrastructures that can integrate and scale out existing access libraries using Ceph’s extensible object model, avoiding reimplementation or even modifications of these access libraries as much as possible. These programmable storage extensions coupled with our distributed dataset mapping techniques enable: 1) access library operations to be offloaded to storage system servers, 2) the independent evolution of access libraries and storage systems and 3) fully leveraging of the existing load balancing, elasticity, and failure management of distributed storage systems like Ceph. They also create more opportunities to conduct storage server-local optimizations specific to storage servers. For example, storage servers might include local key/value stores combined with chunk stores that require different optimizations than a local file system. As storage servers evolve to support new storage devices like non-volatile memory, these server-local optimizations can be implemented while minimizing disruptions to applications. We will report progress on the means by which distributed dataset mapping can be abstracted over particular access libraries, including access libraries for ROOT data, and how we address some of the challenges revolving around data partitioning and composability of access operations.

        Speaker: Jeff LeFevre (University of California, Santa Cruz)
    • 16:30 18:00
      Track 5 – Software Development: Testbeds and CI systems Riverbank R2

      Riverbank R2

      Adelaide Convention Centre

      Convener: Jason Webb (Brookhaven National Lab)
      • 16:30
        The Scalable Systems Laboratory: a Platform for Software Innovation for HEP 15m

        The Scalable Systems Laboratory (SSL), part of the IRIS-HEP Software Institute, provides Institute participants and HEP software developers generally with a means to transition their R&D from conceptual toys to testbeds to production-scale prototypes. The SSL enables tooling, infrastructure, and services supporting innovation of novel analysis and data architectures, development of software elements and tool-chains, reproducible functional and scalability testing of service components, and foundational systems R&D for accelerated services developed by the Institute. The SSL is constructed with a core team having expertise in scale testing and deployment of services across a wide range of cyberinfrastructure. The core team embeds and partners with other areas in the Institute, and with LHC and other HEP development and operations teams as appropriate, to define investigations and required service deployment patterns, to design concrete tests, and to execute and evaluate the results. We describe experiences with early deployments supporting the development of analysis platforms and intelligent data delivery systems.

        Speaker: Robert William Gardner Jr (University of Chicago (US))
      • 16:45
        Fast distributed compilation and testing of large C++ projects 15m

        High energy physics experiments traditionally have large software codebases primarily written in C++ and the LHCb physics software stack is no exception. Compiling from scratch can easily take 5 hours or more for the full stack even on an 8-core VM. In a development workflow, incremental builds are often not sufficient for quick compilation on a typical PC (e.g. due to changes to headers or rebasing), and large shared servers are not practical as users have no control and maintenance is an issue. Even though support for building partial checkouts on top of published project versions exists, by far the most practical development workflow involves full project checkouts because of off-the-shelf tool support (git, intellisense, etc.)

        This paper details a deployment of distcc, a distributed compilation server, on opportunistic resources such as development machines. The best performance operation mode is achieved when preprocessing remotely and profiting from the shared CernVM File System. A 10 (30) fold speedup of elapsed (real) time is achieved when compiling Gaudi, the base of the LHCb stack, when comparing local compilation on a 4 core VM to remote compilation on 80 cores, where the bottleneck becomes non-distributed work such as linking. Compilation results are cached locally using ccache, allowing for even faster rebuilding. A recent distributed memcached-based shared cache is tested as well as a more modern distributed system by Mozilla, sccache, backed by S3 storage. These allow for global sharing of compilation work, which can speed up both central CI builds and local development builds. Finally, we explore remote caching and execution services based on Bazel, and how they apply to Gaudi-based software for distributing not only compilation but also linking and even testing.

        Speaker: Rosen Matev (CERN)
      • 17:00
        Assessing Software Defect Prediction on WLCG Software: a Study with Unlabelled Datasets and Machine Learning Techniques 15m

        Software defect prediction aims at detecting part of software that can likely contain faulty modules - e.g. in terms of complexity, maintainability, and other software characteristics - and therefore that require actual attention. Machine Learning (ML) has proven to be of great value in a variety of Software Engineering tasks, such as software defects prediction, also in the presence of unlabelled datasets that contain a set of features (i.e. software metrics) for the various software modules (such as files, classes and functions) but lack of modules classification like their defectiveness. To accomplish these tasks, datasets have to be collected for the various modules and properly preprocessed before the application of ML techniques: these activities are essential to manage missing values and/or removal inconsistencies amongst data and to make labelled datasets.

        Unlabelled datasets represent the vast majority of software datasets. The extraction of the complete set of features (defectiveness included) and the labeling of the various modules imply effort and time. In literature there exist various approaches to build a prediction model on unlabelled datasets that entail a high number of permutations. Cloud computing infrastructure, GPU-equipped resources and adequate ML framework can give the chance to build software defect prediction model within a reasonable computation time.

        This new study describes the analysis of new unlabelled datasets from WLCG software, coming from HEP-related experiments and middleware, with ML techniques by implementing models in different available frameworks, such as Weka, R and python-based frameworks. We have evaluated these frameworks by considering four aspects: learning curve, extensibility, hardware utilization and speed. This study also includes new approaches to label the various modules due to the heterogeneity of software metrics distribution. Our results suggest that predictive accuracy is generally above 96%; furthermore, our procedure keeps trace of the predict defective modules.

        A major objective of this work is to reduce the distance between theory and practice in software quality, by providing strengths and limitations of the considered frameworks and methods. This will enable developers in WLCG and other scientific communities to assess the applicability of this study to other software, with the ultimate goal to better understand and reduce software defects in complex projects.

        Speaker: Barbara Martelli (INFN CNAF)
      • 17:15
        ART - ATLAS Release Tester using the Grid 15m

        The ART system is designed to run test jobs on the Grid after an ATLAS nightly release has been built. The choice was taken to exploit the Grid as a backend as it offers a huge resource pool, suitable for a deep set of integration tests, and running the tests could be delegated to the highly scalable ATLAS production system (PanDA). The challenge of enabling the Grid as a test environment is met through the use of the CVMFS file system for the software and input data files. Test jobs are submitted to the Grid by the gitlab-ci system which itself is triggered at end of a release build. Jobs can be adorned with special headers that direct the system how to run the specific test, allowing many options to be customised. The gitlab-ci waits for exit status and output files and copied back from the Grid to an EOS area accessible by the users. All gittlab-ci jobs run in ART's virtual machines using docker images for their ATLAS setup. ART jobs can be tracked by using the PanDA system. ART can also be used to run short test jobs locally. It uses the same art command-line interface, where the back-end is replaced to access a local machine for job submission rather than the Grid. This allows developers to ensure their tests work correctly before adding them to the system. In both the Grid and local machine options, running and result copying is completely parallelized. ART is written in python, complete with its own local and Grid tests to give approximately 90% code coverage of the ART tool itself. ART is in production since one year now and full replaces and augments the former system.

        Speaker: Walter Lampl (University of Arizona (US))
      • 17:30
        Evolution of the LHCb Continuous Integration system 15m

        The physics software stack of LHCb is based on Gaudi and is comprised of about 20 interdependent projects, managed across multiple Gitlab repositories. At present, the continuous integration (CI) system used for regular building and testing of this software is implemented using Jenkins and runs on a cluster of about 300 cores.

        LHCb CI pipelines are python-based and relatively modern with some degree of modularity, i.e. the separation of test jobs from build jobs. However, these still suffer from obsoleted design choices that prevent improvements to scalability and reporting. In particular, the resource use and speed have not been thoroughly optimized due to the predominant use of the system for nightly builds, where a feedback time of 8 hours is acceptable. We describe recent work on speeding up pipelines by aggressively splitting and parallelizing checkout, build and test jobs and caching their artifacts. The current state of automatic code quality integration, such as coverage reports, is shown.

        This paper presents how feedback time from change (merge request) submission to build and test reports is reduced from "next day" to a few hours by dedicated on-demand pipelines. Custom GitLab integration allows easy triggering of pipelines, including linked changes to multiple projects, and provides immediate feedback as soon as ready. Reporting includes a comparison to tests on a unique stable reference build, dynamically chosen for every set of changes under testing. This work enables isolated testing of changes that integrates well into the development workflow, leaving nightly testing primarily for integration tests.

        Speaker: Dr Robert Andrew Currie (The University of Edinburgh (GB))
      • 17:45
        Modernizing the CMS software stack 15m

        The CMS experiment relies on a substantial C++ and Python-based software release for its day-to-day production, operations and analysis needs. While very much under active development, this codebase continues to age. At the same time, CMSSW codes are likely to be used for the next two decades, in one form or another. Thus, the "cost" of bugs entering CMSSW continues to increase, both due to the increasing scale of production and analysis activities and due to the increased time to identify bugs in a legacy system. The software integration system for CMSSW must therefore continue to evolve to include modern tools, to provide developers with better information for debugging, and to give code reviewers more tools for code quality evaluation. This talk will describe the approach and results from recent changes to CMSSW including the addition of code formatting checks, the integration of accelerators into build and test procedures, and changing the CMS software base and development platform from Python2 to Python3. We will discuss the continued enhancement of the CMS continuous integration systems based on GitHub and Jenkins as well as the adoption of new code analytics tools.

        Speaker: Mircho Nikolaev Rodozov (Bulgarian Academy of Sciences (BG))
    • 16:30 18:00
      Track 6 – Physics Analysis: Machine Learning for data analysis Hall G

      Hall G

      Adelaide Convention Centre

      Convener: Maurizio Pierini (CERN)
      • 16:30
        Utilizing Machine Learning In BSM Physics Searches At The LHC 15m

        Recent searches for supersymmetric particles at the Large Hadron Collider have been unsuccessful in detecting any BSM physics. This is partially because the exact masses of supersymmetric particles are not known, and as such, searching for them is very difficult. The method broadly used in searching for new physics requires one to optimise on the signal being searched for, potentially suppressing new physics which may actually be present that does not resemble the chosen signal. The problem being that in order to detect something with this method, one must already know what to look for. I will show that a variety of machine-learning techniques can be used to define a "signal-agnostic'' search. This is a search that does not make any assumptions about the signal being searched for, allowing it to detect a signal in a more general way. This method is applied to simulated top-squark data and the results are explored.

        Speaker: Adam Leinweber (Univeristy of Adelaide)
      • 16:45
        ML Track Fitting in Nuclear Physics 15m

        Charged particle tracking represents the largest consumer of CPU resources in high data volume Nuclear Physics experiments. An effort is underway to develop ML networks that will reduce the resources required for charged particle tracking. Tracking in NP experiments represent some unique challenges compared to HEP. In particular, track finding typically represents only a small fraction of the overall tracking problem in NP. This presentation will outline the differences and similarities between NP and HEP charged particle tracking and areas where ML learning may provide a benefit. The status of the specific effort taking place at Jefferson Lab will also be shown.

        Speaker: Thomas Britton (JLab)
      • 17:00
        Reinforcement learning for sorting jets in top pair associated Higgs boson production 15m

        For physics analyses with identical final state objects, e.g. jets, the correct sorting of the objects at the input of the analysis can lead to a considerable performance increase.

        We present a new approach in which a sorting network is placed upstream of a classification network. The sorting network combines the whole event information and explicitly pre-sorts the inputs of the analysis. Because the optimal order is generally not known, a reinforcement learning approach is chosen, in which the sorting agent is trained with end-to-end feedback from the analysis environment. In this way, we enable the system to autonomously find an optimal solution. This new approach works for almost any analysis.

        Using the example of top-quark pair associated Higgs boson production, we show an improvement of the signal and background separation in comparison to conventional sorting of jets with respect to their transverse momenta.

        Speaker: Mr Dennis Noll (RWTH Aachen University (DE))
      • 17:15
        Identification of new long-lived particle states using deep neural networks 15m

        We present preliminary studies of a deep neural network (DNN) "tagger" that is trained to identify the presence of displaced jets arising from the decays of new long-lived particle (LLP) states in data recorded by the CMS detector at the CERN LHC. Particle-level candidates, as well as secondary vertex information, are refined through the use of convolutional neural networks (CNNs) before being combined with high-level engineered variables via a dense neural network. The LLP lifetime is a parameter of the neural network model, which allows for hypothesis testing over several orders of magnitude in lifetime, from ctau = 1 um to 10 m. The network training is performed by streaming ROOT trees containing O(100M) jets directly into the TensorFlow queue and threading system. This custom workflow allows a flexible selection of input features and the asynchronous preprocessing of data, such as the resampling and shuffling of batches on the CPU, in parallel to training on the GPU. Domain adaptation is performed with CMS control region data to ensure that MC simulation can be used to accurately model candidate signal events in data. The technical implementation, architecture, workflows, and potential applications are described. We will demonstrate the tagger performance as evaluated with both MC simulation and control region data. Preliminary studies also demonstrate the potential for a universal, model-independent performance for a broad range of signal model hypotheses, such as long-lived resonances.

        Speaker: Matthias Komm (Imperial College (GB))
      • 17:30
        Physics Inspired Deep Neural Networks for Top Quark Reconstruction 15m

        Deep neural networks (DNNs) have been applied to the fields of computer vision and natural language processing with great success in recent years. The success of these applications has hinged on the development of specialized DNN architectures that take advantage of specific characteristics of the problem to be solved, namely convolutional neural networks for computer vision and recurrent neural networks for natural language processing. This research explores whether a neural network architecture specific to the task of identifying t→Wb decays in particle collision data yields better performance than a generic, fully-connected DNN. This approach is inspired by an DNN technique for tagging boosted top quarks, which consists of defining custom neural network layers known as the combination and Lorentz layers. These layers encode knowledge of relativistic kinematics applied to combinations of particles, and the output of these specialized layers can then be fed into a fully connected neural network to learn tasks such as classification. This research compares the performance of these physics inspired networks to that of a generic, fully-connected DNN, to see if there is any advantage in terms of classification performance, size of the network, or ease of training.

        Speaker: Mr Kevin Greif (University of Notre Dame)
      • 17:45
        Machine learning for QCD theory and data analysis 15m

        We describe a multi-disciplinary project to use machine learning techniques based on neural networks (NNs) to construct a Monte Carlo event generator for lepton-hadron collisions that is agnostic of theoretical assumptions about the microscopic nature of particle reactions. The generator, referred to as ETHER (Empirically Trained Hadronic Event Regenerator), is trained to experimental data along with dedicated detector simulators in order to map out faithfully the multi-particle distributions at femtometer scales. We will discuss how the resulting generator can be a useful tool for the QCD theory and experimental communities. As a further application of machine learning, we present new strategies based on NNs for QCD global analyses where NNs are trained to map the parameter space for the underlying quantum probability distributions into experimental observables. Such mappings will facilitate online visualizations of the correlations between experimental data space and the space of possibilities for the quantum probability distributions. The resulting product will be a critical tool for the nuclear and high energy physics communities, opening up new possibilities for collaboration with computer science in the exploration and visualization of the inner structure of hadrons and nuclei.

        Speakers: Dr Nobuo Sato (Jefferson Lab), nobuo sato (Florida State University)
    • 16:30 18:00
      Track 7 – Facilities, Clouds and Containers: Containers Riverbank R7

      Riverbank R7

      Adelaide Convention Centre

      Convener: Sang Un Ahn (Korea Institute of Science & Technology Information (KR))
      • 16:30
        Using Kubernetes as an ATLAS computing site 15m

        In recent years containerization has revolutionized cloud environments, providing a secure, lightweight, standardized way to package and execute software. Solutions such as Kubernetes enable orchestration of containers in a cluster, including for the purpose of job scheduling. Kubernetes is becoming a de facto standard, available at all major cloud computing providers, and is gaining increased attention from some WLCG sites. In particular, CERN IT has integrated Kubernetes into their cloud infrastructure by providing an interface to instantly create Kubernetes clusters. Also, the University of Victoria is pursuing an infrastructure-as-code approach to deploying Kubernetes as a flexible and resilient platform for running services and delivering resources.
        ATLAS has partnered with CERN IT and the University of Victoria to explore and demonstrate the feasibility of running an ATLAS computing site directly on Kubernetes, replacing all grid computing services. We have interfaced ATLAS’ workload submission engine PanDA with Kubernetes, to directly submit and monitor the status of containerized jobs. This paper will describe the integration and deployment details, and focus on the lessons learned from running a wide variety of ATLAS production payloads on Kubernetes using clusters of several thousand cores at CERN and the Tier 2 computing site in Victoria.

        Speaker: Fernando Harald Barreiro Megino (University of Texas at Arlington)
      • 16:45
        Deployment of containers on the diverse ATLAS infrastructure 15m

        We will describe the deployment of containers on the ATLAS infrastructure. There are several ways to run containers: as part of the batch system infrastructure, as part of the pilot, or called directly. ATLAS is exploiting them depending on which facility its jobs are sent to. Containers have been a vital part of the HPC infrastructure for the past year, and using fat images - images containing several releases and other data - created using the cvmfs_shrinkwrap utility has allowed ATLAS to run production jobs as part of the HPC infrastructure at places like NeRSC and produce 100s millions events. At other sites using non-redhat-based linux distributions, ATLAS could run thanks to containers embedded in the batch system infrastructure and using the ADC images in CVMFS, which are also used at BOINC sites. To run more extensively at all grid sites, we have devised and integrated in the new ATLAS pilot two ways to deploy containers: one wrapping them in ALRB (AtlasLocalRootBase), which is how ATLAS set up the environment, and the other is using standalone containers that can run code independently from the environment they are executed in. Both methods respond to different use cases; the first one will be completely transparent to the user and production teams, while the second will put the user in a position to choose the software to run and how to run it. The two methods also meet the requirement of running on a diverse range of sites from standard grid sites, to cloud sites, to HPC sites with no network and different architectures; in particular, it opened the possibility to use hardware accelerated workflows at grid sites and in the future at HPC sites. Access to the images for both methods is handled differently since ALRB containerization relies on images being distributed via CVMFS, while standalone containers can be more integrated in a CI workflow, which needs a faster turnaround of the images on registries such as docker and gitlab during development, or can be used as a software distribution method for networkless sites. Methods to robustly transfer and manage a large number of popular user images from the registries to CVMFS are under evaluation. We will describe different setups required at diverse sites to achieve payload isolation from the pilot environment required to comply with the WLCG traceability and isolation policy. To be able to adapt to the evolving containers ecosystem and the sites requests, we will also describe how the flexibility to use different runtimes, other than singularity, has been incorporated in the infrastructure.

        Speaker: Alessandra Forti (University of Manchester (GB))
      • 17:00
        Science Box: Converging to Kubernetes containers in production for on-premise and hybrid clouds for CERNBox, SWAN, and EOS 15m

        Container technologies are rapidly becoming the preferred way by developers and system administrators to package applications, distribute software and run services. A crucial role is played by container orchestration software such as Kubernetes, which is also the natural fit for microservice-based architectures. Complex services are re-thought as a collection of fundamental applications (each of these hosted in a separate container), while the final service is achieved by executing multiple containers at the same.

        The Storage Group of the IT department at CERN has been successfully exploiting containers technology as a basis of Science Box, a self-contained Docker-based version of EOS (the CERN storage technology for LHC data and users' files), CERNBox (cloud synchronization and sharing for science), and SWAN (Service for Web-based ANalysis). Science Box has been successfully deployed on multiple cloud providers, including commercial platforms such as Amazon or Open Telekom Cloud.

        In 2018, Science Box was at the core of a project investigating Big Data tools to analyze data from the TOTEM experiment at the LHC. In this context, a Kubernetes-managed instance of EOS, CERNBox, and SWAN has been deployed on the infrastructure provided by the Helix Nebula Science Cloud, an initiative targeting procurement of cloud resources from commercial providers and publicly funded science clouds. The infrastructure consisted of 400 CPUs, 1.5TB of memory, and 22TB of block storage. In addition, SWAN has been interfaced with a dedicated Spark cluster of ~2,000 cores to boost its computational capabilities. The deployment has been maintained for approximately 6 months during which it has been actively used by the TOTEM scientists to exploit a new interface for declarative analysis (called RDataFrame and now part of the ROOT analysis framework), which enables interactive processing of large datasets. The system provided validated physics results and achieved considerable speed-ups, effectively allowing the physicists to perform complex analysis tasks in quasi-interactive response times.

        We are currently investigating the feasibility of running critical production storage services at CERN in containers. We leverage the experience gained with the development of Science Box and plan to evolve our service deployment model by combining Kubernetes to orchestrate containers execution and Helm to manager their configuration. In addition, we plan to use the cloud container orchestration service provided by the Computing and Monitoring group of CERN-IT, which employs OpenStack-provided resources and embeds centralized monitoring and auto-scaling capabilities.

        Speaker: Enrico Bocchi (CERN)
      • 17:15
        Managing the CERN Batch System with Kubernetes 15m

        The CERN Batch Service faces many challenges in order to get ready for the computing demands of future LHC runs. These challenges require that we look at all potential resources, assessing how efficiently we use them and that we explore different alternatives to exploit opportunistic resources in our infrastructure as well as outside of the CERN computing centre.

        Several projects, like BEER, Helix Nebula Science Cloud and the new OCRE project, have proven our ability to run batch workloads on a wide range of non-traditional resources. However, the challenge is not only to obtain the raw compute resources needed but how to define an operational model that is cost and time efficient, scalable and flexible enough to adapt to a heterogeneous infrastructure.

        In order to tackle both the provisioning and operational challenges it was decided to use Kubernetes. By using Kubernetes we benefit from a de-facto standard in containerised environments, available in nearly all cloud providers and surrounded by a vibrant ecosystem of open-source projects. Leveraging Kubernetes’ built-in functionality, and other open-source tools such as Helm, Terraform and GitLab CI, we have deployed the first cluster prototype which we discuss in detail. The effort has simplified many of the existing operational procedures we currently have, but has also made us rethink many established procedures and assumptions that were only valid in a VM based cloud environment.

        This contribution presents how we have adopted Kubernetes in the CERN Batch Service, the impact its adoption has had in daily operations, a comparison on resource usage efficiency and the experience so far evolving our infrastructure towards this model.

        Speaker: Luis Fernandez Alvarez (CERN)
      • 17:30
        Abstracting container technologies and transfer mechanisms in the Scalable CyberInfrastructure for Artificial Intelligence and Likelihood Free Inference (SCAILFIN) project 15m

        High Performance Computing (HPC) facilities provide vast computational power and storage, but generally work on fixed environments designed to address the most common software needs locally, making it challenging for users to bring their own software. To overcome this issue, most HPC facilities have added support for HPC friendly container technologies such as Shifter, Singularity, or CharlieCloud. These different container technologies are all compatible with the more popular Docker containers, however the implementation and use of said containers is different for each HPC friendly container technology. These usage differences can make it difficult for an end user to easily submit and utilize different HPC sites without making adjustments to their workflows and software. This issue is exacerbated when attempting to utilize workflow management software between different sites with differing container technologies.

        The SCAILFIN project aims to develop and deploy artificial intelligence (AI) and likelihood-free inference (LFI) techniques and software using scalable cyberinfrastructure (CI) that span multiple sites. The project has extended the CERN-based REANA framework, a platform designed to enable analysis reusability, and reproducibility while supporting different workflow engine languages, in order to support submission to different HPC facilities. The work presented here focuses on the development of an abstraction layer that allows the support of different container technologies and different transfer protocols for files and directories between the HPC facility and the REANA cluster edge service from the user's workflow application.

        Speaker: Kenyi Paolo Hurtado Anampa (University of Notre Dame (US))
      • 17:45
        Running ALICE Grid Jobs in Containers - A new approach to job execution for the next generation ALICE Grid framework 15m

        The new jAliEn (Java ALICE Environment) middleware is a Grid framework designed to satisfy the needs of the ALICE experiment for the LHC Run 3, such as providing a high-performance and high-scalability service to cope with the increased volumes of collected data. This new framework also introduces a split, two-layered job pilot, creating a new approach to how jobs are handled and executed within the Grid. Each layer runs on a separate JVM, with a separate authentication token, allowing for a finer control of permissions and improved isolation of the payload. Having these separate layers also allows for the execution of job payloads within containers. This allows for the further strengthening of isolation and creates a cohesive environment across computing sites, while avoiding the resource overhead associated with traditional virtualisation.
        This contribution presents the architecture of the new split job pilot found in jAliEn, and the methods used to achieve the execution of Grid jobs while maintaining reliable communication between layers. Specifically, how this is achieved despite the possibility of a layer being run in a separate container, while retaining isolation and mitigating possible security risks. Furthermore, we discuss how the implementation remains agnostic to the choice of container platform, allowing it to run within popular platforms such as Singularity and Docker.

        Speaker: Maksim Melnik Storetvedt (Western Norway University of Applied Sciences (NO))
    • 16:30 18:00
      Track 8 – Collaboration, Education, Training and Outreach: Web Riverbank R1

      Riverbank R1

      Adelaide Convention Centre

      Convener: Ilya Komarov
      • 16:30
        EVE-7 and FireworksWeb: The next generation event visualization tools for ROOT and CMS 15m

        The CMS experiment supports and contributes the development of next-generation Event Visualization Environment (EVE) of the ROOT framework with the intention of superseding Fireworks, the physics analysis oriented event display of CMS that was developed ten years ago and has been used for Run 1 and Run 2, with a new server-web client implementation. This paper presents progress in development of EVE-7 visualization library and FireworksWeb prototype application.

        EVE-7 is a rewrite of EVE for the ROOT-7 era, using modern C++ and relying on ROOTs built-in http server for communication with GUI clients. Part of EVE-7 is also implemented in JavaScript and uses OpenUI5, JSROOT, and Three.js as its foundation libraries.

        FireworksWeb is currently at the stage of a minimal application built around EVE-7. Several advanced Fireworks features have been ported into EVE-7 in an experiment-independent manner, relying heavily on Cling, the C++ interpreter of ROOT: dynamic table views, handling of physics object collections, and filtering of objects within physics collections.

        Examples from EVE-7 and FireworksWeb will be given to explain and to demonstrate the building of graphical scenes and table views from physics collections, event navigation, and the selection and highlight of physics objects and all their representations.

        Speaker: Matevz Tadel (Univ. of California San Diego (US))
      • 16:45
        The official website of the AMS experiment 15m

        The Alpha Magnetic Spectrometer (AMS) is a particle physics experiment installed and operating on board of the International Space Station (ISS) from May 2011 and expected to last through Year 2024 and beyond. Aiming to explore a new frontier in particle physic, the AMS collaboration seeks to store, manage and present its research results as well as the details of the detector and the operations through an official website. An open-source content management framework is employed as the platform to build the website, which allows to manage a variety of information, e.g., institutes in the collaboration, physics results, publications, academic events, etc. The motivation of the website, the development strategy, the infrastructure, the web design concepts and technics, the development of the custom modules, and the strategy and implementation of the science data sharing and preservation are discussed.

        Speaker: Jiahui Wei (Universite de Geneve (CH))
      • 17:00
        ATLAS Public Web Site: Evolution of design and content to Drupal 8 15m

        Four years after deployment of our public web site using the Drupal 7 content management system, the ATLAS Education and Outreach group is in the process of migrating to the new CERN Drupal 8 infrastructure. We present lessons learned from the development, usage and evolution of the original web site, and how the choice of technology helped to shape and reinforce our communication strategy. We then discuss tactics for the migration to Drupal 8, including our choice to use the CERN Override theme. This theme was developed by the CERN web team to support clients like ATLAS to develop web sites in the relatively complex and non-intuitive environment of Drupal. Furthermore, CERN has encouraged usage of this theme to mitigate support and ease future migration. We present the effects this choice has had on the design, implementation, operation and maintenance of the new site.

        Keywords: Outreach, Education, Web, Development, Drupal

        Speaker: Steven Goldfarb (University of Melbourne (AU))
      • 17:15
        Promoting HEP Research-Based Education through Undergraduate Research Experience for Students 15m

        Abstract
        Various studies have shown the crucial and strong impact that undergraduate research has on the learning outcome of students and its role in clarifying their career path. It was proven that promoting research at the undergraduate level is essential to build an enriched learning environment for students [1,2]. Students get exposed to the research world at an early stage, acquire new skills, learn about fundamental topics in depth, develop a professional identity and have a clearer idea about the career path they want to follow later. Undergraduate research enhances engaged learning and therefore is considered as a high-impact educational practice to achieve excellence.
        The High Energy and Medical Physics Group at TAMUQ has been supporting and engaging undergraduate students in different research projects in the areas of High Energy Physics and Medical Physics for the past six years. It attracted students more than 25 undergraduate students. Many projects conducted within the group were awarded by the Qatar National Research Fund which is a governmental funding body that provides funding to highly competitive projects that address national priorities and contribute to capacity building [3]. Students were trained to use a high performance computing facility, different HEP programming languages, software and Monte Carlo based platforms for their simulation. As for the outcomes, they participated and presented at international conferences, many of them attended CERN summer internship program and took part in different hands-on activities within the Compact Muon Solenoid (CMS) experiment at CERN. Some of the results produced by students were even published in scientific journals [4,5]. Several students are now enrolled in World Class universities for their master and PhD studies.
        These activities have laid the foundation of a strong and young group of researchers in Qatar.

        In this presentation, we highlight some of the various HEP projects that our students completed, the different tools that were used, as well as the research outcomes. Then, we will discuss the impact of this experience on their learning and undergraduate education as well as their career path, especially their postgraduate studies.

        References

        [1] Desai KV, Gatson SN, Stiles TW, Stewart RH, Laine GA, Quick CM. Integrating research and education at research-extensive universities with research-intensive communities. Adv Physiol Educ. 2008;32(2):136-41.
        [2] Heidi A. Wayment, K. Laurie Dickson. Increasing student participation in undergraduate research benefits students, faculty, and department. Teaching of Psychology 2008;35:194-97.
        [3] The Qatar National Research Fund, http://www.qnrf.org
        [4] M. Abi Akl et al., Uniformity studies in large area triple-GEM based detectors, Nuclear Instruments and Methods A832 (2016)
        [5] O. Bouhali et al., “Accelerating avalanche simulation in gas based charged particle detectors », Nucl. Inst. Method. 901(2018) 92.

        Speaker: Othmane Bouhali (Texas A & M University (US))
      • 17:30
        Rebuilding INSPIRE together with the HEP community 15m

        The INSPIRE digital library serves the scientific community since almost 50 years. Previously known as SPIRES, it was the first web site outside Europe and the first database on the web. Today, INSPIRE connects 100'000 scientists in High Energy Physics worldwide, with over 1 million scientific articles, thousands scientific profiles of authors, data, conferences and jobs in High Energy Physics.

        In order to bring INSPIRE to the next level, recently we rebuilt the platform based on modern infrastructure. We released the new INSPIRE version in beta a few months ago.

        For a successful beta release, we worked closely with the High Energy Physics community to identify the users' needs, drivers and barriers.

        In this paper, we describe the user-driven process that we followed, our testing strategy, as well as the user feedback following the INSPIRE beta release.

        Speaker: Stella Christodoulaki (CERN)
      • 17:45
        The IPPOG Resource Database - Making Particle Physics Outreach & Education Available Worldwide 15m

        The International Particle Physics Outreach Group (IPPOG) is a network of scientists, science educators and communication specialists working across the globe in informal science education and outreach for particle physics. Members initiate, develop and participate in a variety of activities in classrooms, public events, festivals, exhibitions, museums, institute open days, etc. The IPPOG Resource Database was designed to provide a central location hosting written and multimedia material, tools, recipes, and other content to serve these activities. Objects in the database are categorised by language, target audience, age group, media type, and physics topics. Current efforts include porting the database to the new CERN web infrastructure and improving the interface to serve a public audience. We present our efforts to update procurement and categorisation of educational content, in addition to implementation of the redesigned user interface.

        Speaker: Marzena Lapka (CERN)
    • 16:30 18:00
      Track X – Crossover sessions: Collaborative and common software Riverbank R4

      Riverbank R4

      Adelaide Convention Centre

      Conveners: Paul James Laycock (Brookhaven National Laboratory (US)), Steven Schramm (Universite de Geneve (CH))
      • 16:30
        Running synchronous detector reconstruction in ALICE using declarative workflows 15m

        The ALICE experiment at the Large Hadron Collider (LHC) at CERN will deploy a combined online-offline facility for detector readout and reconstruction, as well as data compression. This system is designed to allow the inspection of all collisions at rates of 50 kHz in the case of Pb-Pb and 400 kHz for pp collisions in order to give access to rare physics signals. The input data rate of up to of 3.4 TByte/s requires that a large part of the detector reconstruction will be realized online in the synchronous stage of the system.

        The data processing is based on individual algorithms which will be executed in parallel processes on multiple compute nodes. Data and workload will be distributed among the nodes and processes using message queue communication provided by the FairMQ package of the ALFA software framework. As the ALICE specific layer, a message-passing aware data model and annotation allows to efficiently describe data and routing. Finally, the Data Processing Layer introduces the description of the reconstruction in a data-flow oriented approach, and makes the complicated nature of a distributed system transparent to users and developers. So-called workflows are defined in a declarative language as sequences of processes with inputs, the algorithm, and outputs as the three descriptive properties.

        With this layered structure of the ALICE software, development of specific stages of the reconstruction can be done in a flexible way in the domain of the specified processes without the need of boiler-plate adjustments and taking into account details of the distributed and parallel system. The Data Processing Layer framework takes care of generating the workflow with the required connections and synchronization, and interfaces to the backend deploying the workflow on computing resources. For the development it is completely transparent whether to run a workflow on a laptop or a computer cluster.

        The modular software framework is the basis for splitting the data processing into manageable pieces and helps to distribute development effort. This contribution describes the implementation of reconstruction workflows in the ALICE software framework and the optimization of data objects, and presents results from the realized prototypes.

        Speaker: Matthias Richter (University of Oslo (NO))
      • 16:45
        Let's get our hands dirty: a comprehensive evaluation of DAQDB, key-value store for petascale hot storage. 15m

        Data acquisition (DAQ) systems are a key component for successful data taking in any experiment. The DAQ is a complex distributed computing system and coordinates all operations, from the data selection stage of interesting events to storage elements.
        For the High Luminosity upgrade of the Large Hadron Collider (HL-LHC), the experiments at CERN need to meet challenging requirements to record data with a much higher occupancy in the detectors. The DAQ system will receive and deliver data with a significantly increased trigger rate, one million events per second, and capacity, terabytes of data per second.
        An effective way to meet these requirements is to decouple real-time data acquisition from event selection. Data fragments can be temporarily stored in a large distributed key-value store. Fragments belonging to the same event can be then queried on demand, by the data selection processes.
        Implementing such a model relies on a proper combination of emerging technologies, such as persistent memory, NVMe SSDs, scalable networking, and data structures, as well as high performance, scalable software.
        In this paper, we present DAQDB, an open source implementation of this design that was presented earlier, with an extensive evaluation of this approach, from the single node to the distributed performance. Furthermore, we complement our study with an in-depth comparison with the state-of-the-art solutions, a description of the challenges faced and the lessons learned.

        Speaker: Mr Grzegorz Jereczek (Intel Corporation)
      • 17:00
        Big Data solutions for the online processing of trigger-less detectors data 15m

        The need for an unbiased analysis of large complex datasets, especially those collected by the LHC experiments, is pushing for data acquisition systems where predefined online trigger selections are limited if not suppressed at all. Not just this poses tremendous challenges for the hardware components, but also calls for new strategies for the online software infrastructures. Open source Big-Data tools could certainly offer valuable solutions for the latter.
        In view of the high luminosity upgrade of the LHC, we developed a prototype online processing scheme for the CMS muon detectors (Drift Tubes Chambers, DT), which streams signals from the front-end electronics at the same rate as the LHC clock (40 MHz) and serves them through Apache Kafka to a remote Apache Spark cluster where offline quality reconstruction algorithms are run. Extensive tests have been carried out demonstrating the scalability of the system, in particular the throughput expected from the DT chambers at HL-LHC can be sustained by a computing cluster of comparable size as the current prototype. This setup has been exploited successfully for beam tests and will be deployed for parasitic operations in CMS during the next LHC Run.

        Speaker: Marco Zanetti (Universita e INFN, Padova (IT))
      • 17:15
        Implementation of the ATLAS trigger within the multi-threaded AthenaMT framework 15m

        Athena is the software framework used in the ATLAS experiment throughout the data processing path, from the software trigger system through offline event reconstruction to physics analysis. The shift from high-power single-core CPUs to multi-core systems in the computing market means that the throughput capabilities of the framework have become limited by the available memory per process. For Run 2 of the Large Hadron Collider (LHC), ATLAS has exploited a multi-process forking approach with the copy-on-write mechanism to reduce the memory use. To better match the increasing CPU core count and the, therefore, decreasing available memory per core, a multi-threaded framework, AthenaMT, has been designed and is now being implemented. The ATLAS High Level Trigger (HLT) system has been remodelled to fit the new framework and to rely on common solutions between online and offline software to a greater extent than in Run 2.

        We present the implementation of the new HLT system within the AthenaMT framework, which will be used in ATLAS data-taking during Run 3 (2021-2023) of the LHC.

        Speaker: Teng Jian Khoo (Universite de Geneve (CH))
      • 17:30
        The ACTS project: track reconstruction software for HL-LHC and beyond 15m

        The reconstruction of trajectories of the charged particles in the tracking detectors of high energy physics experiments is one of the most difficult and complex tasks of event reconstruction at particle colliders. As pattern recognition algorithms exhibit combinatorial scaling to high track multiplicities, they become the largest contributor to the CPU consumption within event reconstruction, particularly at current and future hadron colliders such as the LHC, HL-LHC and FCC-hh. Current algorithms provide an extremely high standard of physics and computing performance and have been tested on billions of simulated and recorded data events. However, most algorithms were first written 20 years ago and maintaining them has become increasingly challenging. In addition, they are challenging to adapt to modern programming paradigms and parallel architectures.

        ACTS is based on the well-tested and highly functioning components of LHC track reconstruction algorithms, implemented with modern software concepts and inherently designed for parallel architectures. Multithreading becomes increasingly important to balance the memory usage per CPU core. However, a fully multithreaded event processing framework blurs the clear border between events, which has in the past often been used as a clearly defined validity boundary for event conditions. ACTS is equipped with a full contextual conditions concept that allows to run concurrent track reconstruction even in case of multiple detector alignments, conditions or even varying magnetic field being processed at the same time.
        It provides an experiment and, in particular, framework-independent software core with key tools and light-weight, highly optimised event data model for track reconstruction. Particular care is given to thread safety and data locality. It is designed as a toolbox that allows to implement and extend widely known pattern recognition algorithms, and in addition suitable for algorithm templating and R&D. ACTS has also used as the fast simulation engine for the Tracking Machine Learning Challenge, and will provide reference implementation of several submitted solution programs of the two phases of the challenge.

        We discuss the current status of the ACTS project. We will present studies of the physics and computing performance based on the upcoming first full release. We will discuss challenges in the development of experiment-independent algorithms and illustrate this, by discussing prototype studies of its use in experiments ranging from test-beam to Belle, LHC experiments, CEPC and FCC-hh reference detectors. We will also compare performance between different algorithms, including those based on machine learning.

        Speaker: Paul Gessinger-Befurt (CERN / JGU Mainz)
      • 17:45
        Towards a Turnkey Software Stack for HEP Experiments 15m

        Future HEP experiments require detailed simulation and advanced reconstruction algorithms to explore the physics reach of their proposed machines and to design, optimise, and study the detector geometry and performance. To synergise the development of the CLIC and FCC software efforts, the CERN EP R&D road map proposes the creation of a "Turnkey Software Stack", which is foreseen to provide all the necessary ingredients, from simulation to analysis, for future experiments, beyond CLIC, or FCC to proposed Super-tau-charm factories, CEPC or ILC. The software stack will facilitate writing specific software for experiments ensuring coherency and maximizing re-use of established packages to benefit from existing solutions and community developments, for example, ROOT, Geant4, DD4hep, Gaudi and PODIO. As a showcase for the software stack, the existing CLIC reconstruction software, written for iLCSoft, is being to be ported to Gaudi. In parallel, the back-end of the LCIO event data model can be replaced by an implementation in PODIO. These changes will enable the sharing of the algorithms with other users of the software stack.

        We will present the current status and plans of the turnkey software stack, with a focus of the adaptation of the CLIC reconstruction chain to Gaudi and PODIO, and detail the plans for future developments to generalize their applicability to FCC and beyond.

        Speaker: Andre Sailer (CERN)
    • 09:00 10:30
      Plenary: Accelerated computing Hall G

      Hall G

      Adelaide Convention Centre

      Convener: Caterina Doglioni (Lund University (SE))
      • 09:00
        Meeting the challenge of JUNO simulation with Opticks GPU Optical Photon Acceleration 30m
        Speaker: Simon Blyth
      • 09:30
        Physics performance and event throughput of the GPU High Level Trigger 1 of LHCb 30m

        Beginning in 2021, the upgraded LHCb experiment will use a triggerless readout system collecting data at an event rate of 30 MHz. A software-only High Level Trigger will enable unprecedented flexibility for trigger selections. During the first stage (HLT1), a sub-set of the full offline track reconstruction for charged particles is run to select particles of interest based on single or two-track selections. Track reconstruction at 30 MHz represents a significant computing challenge, requiring an evaluation of the most suitable hardware to be used as well as algorithms optimized for this hardware. In this talk we present work based on an R&D project in the context of the LHCb Upgrade I exploring the approach of executing the full HLT1 on GPUs. This includes decoding the raw data, clustering of hits, pattern recognition, as well as track fitting, ghost track recjection with machine learning techniques and finally event selections. We will discuss the development of algorithms optimized for many-core architectures. Both the physics performance and event throughput of the entire HTL1 application running on GPUs will be presented.

        Speaker: Dorothea Vom Bruch (LPNHE Paris, CNRS)
      • 10:00
        HPC+AI: Accelerating scientific computing for the Exascale era 30m
        Speaker: Jeff Adie (NVIDIA)
    • 10:30 10:50
      Break 20m
    • 10:50 11:50
      Plenary: HPC / Cloud computing Hall G

      Hall G

      Adelaide Convention Centre

      Convener: Paul Douglas Jackson (University of Adelaide)
      • 10:50
        HPC systems in the next decade - what to expect, when, where 30m
        Speaker: Dirk Pleiter (Forschungszentrum Jülich GmbH)
      • 11:20
        Reperforming a Nobel Prize discovery on Kubernetes 30m
        Speakers: Lukas Alexander Heinrich (CERN), Ricardo Brito Da Rocha (CERN)
    • 11:50 12:45
      Lunch 55m
    • 12:45 17:45
      Social Event: Excursions
    • 09:00 10:30
      Plenary: Quantum computing / Security / Gravitational wave astrophysics Hall G

      Hall G

      Adelaide Convention Centre

      Convener: Patrick Fuhrmann (Deutsches Elektronen-Synchrotron (DE))
    • 10:30 11:00
      Break 30m
    • 11:00 12:30
      Track 1 – Online and Real-time Computing: Hardware acceleration and hardware machine learning Riverbank R5

      Riverbank R5

      Adelaide Convention Centre

      Convener: Jennifer Ngadiuba (CERN)
      • 11:00
        Heterogeneous online reconstruction at CMS 15m

        The CMS experiment has been designed with a two-level trigger system: the Level 1 Trigger, implemented on custom-designed electronics, and the High Level Trigger (HLT), a streamlined version of the CMS offline reconstruction software running on a computer farm. A software trigger system requires a trade-off between the complexity of the algorithms running on the available computing resources, the sustainable output rate, and the selection efficiency.

        Industry and HPC have been successfully using heterogeneous computing platforms to achieve higher throughput and better energy efficiency by matching each job to the most appropriate architecture.

        The development of a heterogeneous online reconstruction faces some fundamental challenges: the algorithms must achieve the same or better physics performance and processing throughput; they have to be integrated in the experimental reconstruction framework; and it must be possible to run and validate them on conventional machines, without any dedicated resources.

        Some of the most time consuming algorithms running in the CMS HLT have been ported to run on NVIDIA GPUs using the CUDA platform; the same algorithms can run on conventional CPUs, with identical results and close to naive performance, leveraging "performance portability" libraries.

        This presentation will describe the results of these developments and the characteristics of the system that is aimed for deployment in production starting from Run 3.

        Speaker: Dr Andrea Bocci (CERN)
      • 11:15
        Level-1 track finding with an all-FPGA system at CMS for the HL-LHC 15m

        The CMS experiment at the LHC is designed to study a wide range of high energy physics phenomena. It employs a large all-silicon tracker within a 3.8 T magnetic solenoid, which allows precise measurements of transverse momentum (pT) and vertex position.

        This tracking detector will be upgraded to coincide with the installation of the High-Luminosity LHC, which will provide up to about 10^35 cm^2 /s to CMS, or 200 collisions per 25 ns bunch crossing. This new tracker must maintain the nominal physics performance in this more challenging environment. Novel tracking modules that utilise closely spaced silicon sensors to discriminate on track pT have been developed that would allow the readout of only hits compatible with pT > 2-3 GeV tracks to off-detector trigger electronics. This would allow the use of tracking information at the Level-1 trigger of the experiment, a requirement to keep the Level-1 triggering rate below the 750 kHz target, while maintaining physics sensitivity.

        This talk presents the concept for an all FPGA based track finder using a time-multiplexed architecture. Hardware demonstrators running a selection of algorithms in real-time have been assembled to prove the feasibility and capability of such a system. The performance for a variety of physics scenarios will be presented, as well as the work to scale the demonstrators to the final system, and exploit new technologies.

        Speaker: Thomas Owen James (CERN)
      • 11:30
        Fast and resource-efficient Deep NN on FPGAs for the Phase-II L0 Muon Barrel Trigger of the ATLAS Experiment 15m

        The Level-0 Muon Trigger system of the ATLAS experiment will undergo a full upgrade for HL-LHC to stand the challenging performances requested with the increasing instantaneous luminosity. The upgraded trigger system foresees to send RPC raw hit data to the off-detector trigger processors, where the trigger algorithms run on new generation of Field-Programmable Gate Arrays (FPGAs). The FPGA represents an optimal solution in this context, because of its flexibility, wide availability of logical resources and high processing speed.
        We have developed novel precision deep neural network architectures based on trained ternary quantization, optimised to run on FPGAs, and trained it for efficient reconstruction and identification of muons in the ATLAS level-0 trigger. Both physics performance in terms of efficiency and fake rates, and FPGA logic resource occupancy and timing obtained with the developed algorithms will be presented.

        Speaker: Stefano Giagu (Sapienza Universita e INFN, Roma I (IT))
      • 11:45
        Fast inference using FPGAs for DUNE data reconstruction 15m

        The Deep Underground Neutrino Experiment (DUNE) will be a world-class neutrino observatory and nucleon decay detector aiming to address some of the most fundamental questions in particle physics. With a modular liquid argon time-projection chamber (LArTPC) of 40 kt fiducial mass, the DUNE far detector will be able to reconstruct neutrino interactions with an unprecedented resolution. With no triggering and no zero suppression or compression, the raw data volume for four modules would be of order 145 EB/year. Consequently, fast and affordable reconstruction methods are needed. Several state-of-the-art methods are focused on machine learning (ML) approaches to identify the signal within the raw data or to classify the neutrino interaction during the reconstruction. One of the main advantages of using those techniques is that they will reduce the computational cost and time compared to classical strategies. Our plan aims to go a bit further and test the implementation of those techniques on an accelerator board. In this work, we present the accelerator board used, a commercial off-the-shelf (COTS) hardware for fast deep learning inference based on an FPGA, and the experimental results obtained outperforming more traditional processing units.

        Speaker: Manuel Jesus Rodriguez Alonso (CERN)
      • 12:00
        Highly Performant, Deep Neural Networks with sub-microsecond latency on FPGAs for Trigger Applications 15m

        Artificial neural networks are becoming a standard tool for data analysis, but their potential remains yet to be widely used for hardware-level trigger applications. Nowadays, high-end FPGAs, as they are also often used in low-level hardware triggers, offer theoretically enough performance to allow for the inclusion of networks of considerable size into these system for the first time. This makes it appear very promising and rewarding to optimize a neural network implementation for FPGAs in the trigger context.
        We present a bottom-up approach of implementing neural networks on FPGAs. For this, we analyzed how typical NN layers could have their processing, data flow and controlling implemented to not only take the trigger environment constraints into account, i.e. incoming data rates of up to multiple tens of MHz and sub-microsecond latency limits, but to also make very efficient use of the resources of the FPGA. This allowed us to develop a highly optimized neural network implementation framework, which typically reaches 90 to 100 % computational efficiency, requires few extra FPGA resources for data flow and controlling, and allows latencies in the order of 10s to few 100s of nanoseconds for entire (deep) networks. Among the implemented layers are 2D convolutions and pooling (both with multi-channel support) as well as dense layers, all of which play a role in many physics-/detector-related applications. Significant effort needed to be put especially into the 2D convolutional layers, as a fast and simultaneously resourceful implementation is quite challenging.
        Results are presented for individual layers as well as entire networks. The FPGA implementations of those example networks were automatically generated from trained NN models by our supplementary toolkit, which was built around the optimized layer framework.

        Speaker: Christian Schmitt (Johannes Gutenberg Universitaet Mainz (DE))
      • 12:15
        hls4ml: deploying deep learning on FPGAs for L1 trigger and Data Acquisition 15m

        Machine learning is becoming ubiquitous across HEP. There is great potential to improve trigger and DAQ performance with it. However, the exploration of such techniques within the field in low latency/power FPGAs has just begun. We present hls4ml, a user-friendly software, based on High-Level Synthesis (HLS), designed to deploy network architectures on FPGAs. As a case study, we use hls4ml for boosted-jet tagging with deep networks at the LHC. We map out resource usage and latency versus network architectures, to identify the typical problem complexity that hls4ml could deal with. We discuss current applications in HEP experiments and future applications. We also report on recent progress in the past year on newer neural network architectures and networks with orders of magnitude more parameters.

        Speaker: Vladimir Loncar (University of Belgrade (RS))
    • 11:00 12:30
      Track 2 – Offline Computing: ML Tracking and parallelisation Riverbank R6

      Riverbank R6

      Adelaide Convention Centre

      Convener: Paul James Laycock (Brookhaven National Laboratory (US))
      • 11:00
        GPU Application in JUNO 15m

        The Jiangmen Underground Neutrino Observatory (JUNO) in China is a 20 kton liquid scintillator detector, designed primarily to determine the neutrino mass hierarchy, as well as to study various neutrino physics topics. Its core part consists of O(10^4) Photomultiplier Tubes (PMTs). Computations looping through this large amount of PMTs on CPU will be very time consuming. GPU parallel computing is perfectly suitable for solving this issue. It could be utilized in event reconstruction, Monte Carlo Simulation and many other aspects of the experiment. This talk will show a few examples of GPU application in JUNO and demonstrate its huge potential for experiments with lots of PMTs.

        Speaker: Dr Wuming Luo (Institute of High Energy Physics, CAS)
      • 11:15
        Reconstruction for Liquid Argon TPC Neutrino Detectors Using Parallel Architectures 15m

        Neutrinos are particles that interact rarely, so identifying them requires large detectors which produce lots of data. Processing this data with the computing power available is becoming more difficult as the detectors increase in size to reach their physics goals. In liquid argon time projection chambers (TPCs) the charged particles from neutrino interactions produce ionization electrons which drift in an electric field towards a series of collection wires, and the signal on the wires is used to reconstruct the interaction. The MicroBooNE detector currently collecting data at Fermilab has 8000 wires, and planned future experiments like DUNE will have 100 times more, which means that the time required to reconstruct an event will scale accordingly. Modernization of liquid argon TPC reconstruction code, including vectorization, parallelization and code portability to GPUs, will help to mitigate these challenges. The liquid argon TPC hit finding algorithm within the LarSoft framework used across multiple experiments has been vectorized and parallelized. This increases the speed of the algorithm on the order of ten times within a stand alone version on Intel architectures. This new version has been incorporated back into LarSoft so that it can be generally used. These methods will also be applied to other low level reconstruction of the wire signals such as the deconvolution. The applications and performance of this modernized liquid argon TPC wire reconstruction will be presented.

        Speaker: Giuseppe Cerati (Fermi National Accelerator Lab. (US))
      • 11:30
        Similarity Hashing and Learning for Tracks Reconstruction 15m

        At the High Luminosity Large Hadron Collider (HL-LHC), many
        proton-proton collisions happen during a single bunch crossing. This
        leads on average to tens of thousands of particles emerging from the
        interaction region. Two major factors impede finding charged particle
        trajectories from measured hits in the tracking detectors. First,
        deciding whether a given set of hits was produced by a common particle
        is an under-specified task. State-of-the-art reconstruction models
        usually tackle this issue via so-called track following only at a
        later stage after considering many hits. Second, assuming a nearly
        perfect hit-particle decision function, constructing possible hit
        combinations to their compatibility using this decision function is a
        combinatorial problem. Thus, the traditional approach will grow
        exponentially as the number of simultaneous collisions increase at the
        HL-LHC and pose a major computational challenge.

        We propose a framework for Similarity Hashing and Learning for Track
        Reconstruction (SHLTR) where multiple small regions of the detector
        are reconstructed in parallel with minimal fake rate. We use hashing
        techniques to separate the detector search space into buckets. The
        particle purity of these buckets, i.e. how many hits from the same
        particle are contained, is increased using locality sensitivity in
        feature space where per-hit features beyond just its position are
        considered. The bucket size is sufficiently small to significantly
        reduce the complexity of track reconstruction within the buckets or
        regions.

        A neural network selects valid combinations in the buckets and builds
        up full trajectories by connected components search independently of
        global positions of the hits and detector geometry. The whole process
        occurs simultaneously in the multiple regions of the detector and
        curved particles are found by allowing buckets to overlap. We present
        first results of such a track reconstruction chain including
        efficiency, fake estimates, and computational performances in µ=200
        datasets.

        Speaker: Sabrina Amrouche (Université de Geneve (CH))
      • 11:45
        Mikado approach for the TrackML Particle Tracking Challenge 15m

        The Mikado approach is the winner algorithm of the final phase of the TrackML particle reconstruction challenge [1].

        The algorithm is combinatorial. Its strategy is to reconstruct data in small portions, each time trying to not damage the rest of the data. The idea reminds Mikado game, where players should carefully remove wood sticks one-by-one from a heap.

        The algorithm does 60 reconstruction passes, each time reconstructing only small portion of tracks. A high speed is achieved thanks to a fast data access within fixed-size search windows.

        As the search windows are individual for each reconstruction pass, the algorithm has tens of thousands of parameters to tune. The parameters were trained on the ground truth data, making the Mikado approach similar to Machine Learning approaches.

        The algorithm shows $94.4\%$ accuracy and takes $0.56$ seconds per event.

        [1] Sabrina Amrouche et al., The Tracking Machine Learning challenge: Accuracy phase, NIPS 2018 proceedings.

        Speaker: Sergey Gorbunov (Johann-Wolfgang-Goethe Univ. (DE))
      • 12:00
        Reconstruction of Charged Particle Tracks in Realistic Detector Geometry Using a Vectorized and Parallelized Kalman Filter Algorithm 15m

        One of the most computationally challenging problems expected for the High-Luminosity Large Hadron Collider (HL-LHC) is finding and fitting particle tracks during event reconstruction. Algorithms used at the LHC today rely on Kalman filtering, which builds physical trajectories incrementally while incorporating material effects and error estimation. Recognizing the need for faster computational throughput, we have adapted Kalman-filter-based methods for highly parallel, many-core SIMD and SIMT architectures that are now prevalent in high-performance hardware.

        Previously we observed significant parallel speedups, with physics performance comparable to CMS standard tracking, on Intel Xeon, Intel Xeon Phi, and (to a limited extent) NVIDIA GPUs. While early tests were based on artificial events occurring inside an idealized barrel detector, we showed subsequently that our "mkFit" software builds tracks successfully from complex simulated events (including detector pileup) occurring inside a geometrically accurate representation of the CMS-2017 tracker. Here, we report on advances in both the computational and physics performance of mkFit, as well as progress toward integration with CMS production software (CMSSW).

        Recently we have improved the overall efficiency of the algorithm by preserving short track candidates at a relatively early stage rather than attempting to extend them over many layers. Moreover, mkFit formerly produced an excess of duplicate tracks; these are now explicitly removed in an additional processing step (note that mkFit builds all its tracks in parallel, so it cannot eliminate detector hits from further consideration as viable tracks are found). We demonstrate that with these enhancements, mkFit becomes a suitable choice for the first iteration of CMS tracking, and perhaps for later iterations as well. We plan to test this capability in the CMS High Level Trigger during Run 3 of the LHC, with an ultimate goal of using it in both the CMS HLT and offline reconstruction for the HL-LHC and Phase II CMS tracker.

        Speaker: Giuseppe Cerati (Fermi National Accelerator Lab. (US))
      • 12:15
        Edepillim: A New Muon Energy Reconstruction Method 15m

        Large scale neutrino detectors are relying on accurate muon energy estimates to infer neutrino energy. Reconstruction methods which incorporate physics knowledge will produce a better result. The muon energy reconstruction algorithm Edepillim takes into account the entire pattern of energy loss along the muon track and uses probability distribution functions describing muon energy losses to perform a maximum likelihood reconstruction for the initial muon energy. This work demonstrates the good reconstruction resolution of this method on idealised simulation by comparison to other energy reconstruction methods.

        Speaker: Dr Sally Robertson (University of California Berkeley, Lawrence Berkeley National Lab)
    • 11:00 12:30
      Track 3 – Middleware and Distributed Computing: Infrastructure & Identity Riverbank R3

      Riverbank R3

      Adelaide Convention Centre

      Convener: James Letts (Univ. of California San Diego (US))
      • 11:00
        DIRACOS: a cross platform solution for grid tools 15m

        DIRACOS is a project aimed to provide a stable base layer of dependencies, on top of which the DIRAC middleware is running. The goal was to produce a coherent environment for grid interaction and streamline the operational overhead. Historically the DIRAC dependencies were grouped in two bundles; Externals containing Python and standard binary libraries, and the LCGBundle which contained all grid-related libraries (gfal, arc, etc). Such a setup proved difficult to test and hindered agile development. DIRACOS solves the binary incompatibility that was caused by using a python version newer than the native system one (SLC6). It is spawned form a single list of required packages from where we use SRPMs to pull all dependencies down to the level of glibc. With such an approach we can provide the same packages for our clients, servers, and several platforms. It is an extendible setup with an DevOps development cycle in mind. The core build functionality of DIRACOS is based on Fedora Mock. DIRACOS also introduces its own grammar, to handle specific cases and it also allows patching (some SRPM require tweeking, which the user can do by providing a diff) as well as routines for pre/post/instead actions of compilation. With this approach DIRAC was able to provide a single bundle for clients and servers, that is reliable, flexible, easy to test and relatively small (250 MB). It allows for a smooth transition from SLC6 to CC7 and provides a clear roadmap for possible extension of DIRAC to a wide variety of platforms.

        Speaker: Christophe Haen (CERN)
      • 11:15
        CERN's Identity and Access Management, a journey to Open Source 15m

        Until recently, CERN had been considered eligible for academic pricing of Microsoft products. Now, along with many other research institutes, CERN has been disqualified from this educational programme and faces a 20 fold increase in license costs. CERN’s current Authentication and Authorisation Infrastructure comprises Microsoft services all the way down from the web Single-Sign-On to the Accounts Database. Replacing these core components is an opportunity to rebuild the CERN infrastructure using the latest technologies and concepts, to respond to the evolving requirements of the community. It is also the appropriate moment to consider the alignment of CERN and WLCG’s approach to identity management, to create a more consistent environment for operators, developers and users. 2019 will see the launch of an Alpha version of CERN’s next generation Authentication and Authorisation Infrastructure, focusing on free and open source products and responding to the limitations experienced by the current system. We will describe the new solution, demonstrate workflows and highlight ways in which this work is being, and can be, used by the wider community.

        Speaker: Hannah Short (CERN)
      • 11:30
        Beyond X.509: token-based authentication and authorization in practice 15m

        One of the key challenges identified by the HEP R&D roadmap for software and computing is the ability to integrate heterogeneous resources in support of the computing needs of HL-LHC. In order to meet this objective, a flexible Authentication and Authorization Infrastructure (AAI) has to be in place, to allow the secure composition of computing and storage resources provisioned across heterogeneous providers (e.g., Grid, private and commercial Clouds, HPC centers).

        At CHEP 2018, we presented how a flexible AAI based on modern, standard Web technologies (OpenID Connect, OAuth and JSON Web Tokens (JWTs)) and centered on the INDIGO Identity and Access Management (IAM) service could support the transition of the WLCG infrastructure to a token-based AAI. In the meanwhile, INDIGO IAM has been selected by the WLCG Management Board as the solution that will be adopted by LHC experiments, and is also at the core of the AAI envisioned to support the computing needs of the ESCAPE project.

        In this contribution, which represents a follow up to last-year plenary talk, we describe the work done in the past year on the IAM service, discussing the results of the initial integration activities with key middleware components for data management and computing (StoRM, dCache, DPM, XRootD, RUCIO, HTCondor). We also discuss how relevant standardization efforts (e.g., the WLCG Authorization WG activities and AARC) and emerging requirements from other communities are driving the IAM evolution agenda.

        Speaker: Andrea Ceccanti (Universita e INFN, Bologna (IT))
      • 11:45
        Distributing User Code with the CernVM FileSystem 15m

        The CernVM FileSystem (CVMFS) is widely used in High Throughput Computing to efficiently distributed experiment code. However, the standard CVMFS publishing tools are designed for a small group of people from each experiment to maintain common software, and the tools don't work well for the majority of users that submit jobs related to each experiment. As a result, most user code, such as code to do specific physics analyses, is still sent with jobs. That process is relatively inefficient, especially when the user code is large. To overcome these limitations, we have built a CVMFS user code publication system. Users submit code as a tarball with their jobs, and the job submission client sends the tarball to one of a pair of user code publishing servers. The code is published in one of a small number of shared CVMFS repositories, and updates are expedited through the standard CVMFS infrastructure to be available on worker nodes worldwide within several minutes. A job wrapper automatically passes the location of the code to the jobs. The user code is automatically deleted from CVMFS after a period of no use. Most of the software for the system is available as a single self-contained open source rpm called cvmfs-user-pub, and is available for other deployments.

        Speaker: Kenneth Richard Herner (Fermi National Accelerator Laboratory (US))
      • 12:00
        GOCDB - new communities, new requirements, new architecture 15m

        GOCDB is the official repository for storing and presenting EGI and WLCG topology and resource information. It is a definitive information source, with the emphasis on user communities to maintain their own data. It is intentionally designed to have no dependencies on other operational tools for information.

        In recent years, funding sources and user communities have evolved and GOCDB is developing to meet the resulting new requirements, such as allowing more programmatic updates to the data within GOCDB. We will explain the roadmap for developing GOCDB and the motivation for the changes on that Roadmap.

        As GOCDB, and the team supporting it, has evolved, we have re-examined the underpinning architecture for GOCDB. We will set out the changes we have made as a result.

        Speaker: Greg Corbett (STFC)
      • 12:15
        Migrating INFN-T1 from CREAM-CE/LSF to HTCondor-CE/HTCondor 15m

        The INFN Tier-1 datacentre provides computing resources to several HEP and Astrophysics experiments. These are organized in Virtual Organizations submitting jobs to our computing facilities through Computing Elements, acting as Grid interfaces to the Local Resource Manager. We are phasing-out our current LRMS (IBM/Platform LSF 9.1.3) and CEs (CREAM) set to adopt HTCondor as a replacement for LSF and HTCondor-CE in place of CREAM. A small instance has been set up to practice with the cluster management and evaluate the feasibility of our migration plans to a new LRMS and CE set. A second cluster instance has been setup to work on production. A number of management tools have been adapted or rewritten in order to integrate the new system with the existing infrastructure. Two different accounting solution for the HTCondor-CE have been implemented, and the more reliable one have been adopted. A python tool has been written to disentangle the management of HTCondor machines from our puppet instance, and to enable a quicker configuration of the cluster nodes. The monitoring tools tied to the old system are being adapted to also work on the new one. Finally, the most relevant setup steps have been documented in a public wiki page and a support mailing has been created to help other INFN sites willing to migrate their LRMS and CE to HTCondor. This document reports about our experience with HTCondor-CE on top of HTCondor and the integration of this system into our infrastructure.

        Speaker: Stefano Dal Pra (Universita e INFN, Bologna (IT))
    • 11:00 12:30
      Track 4 – Data Organisation, Management and Access: storage systems Riverbank R8

      Riverbank R8

      Adelaide Convention Centre

      Convener: Brian Paul Bockelman (University of Nebraska Lincoln (US))
      • 11:00
        dCache - keeping up with the evolution of science 15m

        The dCache project provides open-source software deployed internationally
        to satisfy ever more demanding storage requirements of various scientific
        communities. Its multifaceted approach provides an integrated way of supporting different use-cases with the same storage, from high throughput data ingest, through wide access and easy integration with existing systems, including
        event driven workflow management.

        With this presentation, we will show some of the recent developments that
        optimize data management and access to maximise the gain from stored data.

        Speaker: Mr Tigran Mkrtchyan (DESY)
      • 11:15
        Disk Pool Manager(DPM): From DOME to LHC Run-3 15m

        The DOMA activities gave the opportunity for DPM to contribute to
        the WLCG plans for Run-3 and beyond. Here we identify the themes
        that are relevant to site storage systems and explain how the
        approaches chosen in DPM are relevant for features like
        scalability, third party copy, bearer tokens, multi-site deployments and
        volatile caching pools.

        We will also discuss the status of the project, in relation to the
        technical milestones that allowed important evolution steps, in the
        direction of higher quality, maintainable components like DOME and
        dmlite-shell.

        Speaker: Francesco Giovanni Sciacca (Universitaet Bern (CH))
      • 11:30
        CDFS: A high-efficiency Data Access System for Storage Federations 15m

        High energy physics (HEP) experiments produce a large amount of data, which is usually stored and processed on distributed sites. Nowadays, the distributed data management system faces some challenges such as global file namespace and efficient data access. Focusing on those problems, the paper proposed a cross-domain data access file system (CDFS), a data cache and access system across distributed sites based on edge computing model, using flexible data caching and synchronization, applying data deduplication and compression, aiming at dynamically building an aggregate view of multiple distributed storage and accessing data in a fast and efficient way.
        The CDFS system consists of metadata server, cache server, storage-optimized engine, and data access interface. Metadata server locally builds a very fast dynamic namespace from multiple sites that expose protocols such as Xrootd, HTTP and S3, covering the real file location. Cache server caches and synchronizes file content and metadata on-demand, speeding up data access and directory organization. Storage-optimized engine includes deduplication and compression. Deduplication assure that only nonexistent data block can be transferred to the site, eliminating redundant storage of the same data blocks at one site; compression makes data blocks stored after being compressed, minimizing the space that one data block required. The data access interface provides a command line and a FUSE client for users to access data in a convenient way, hiding the complexity of the transfer process.
        The test results based on the raw data of LHAASO experiment showed that the CDFS could present a unique repository based on distributed data in the sites of Chengdu, Daocheng and Beijing. In addition, the caching mechanism leads to a more than 10 times improvement in data access performance, while the storage-optimized engine reduces the storage consumption of the raw data by about 50%.

        Speakers: Shiyuan Fu, Shiyuan Fu (Institute of High Energy Physics,Chinese Academy of Sciences)
      • 11:45
        CERN Tape Archive: production status, migration from CASTOR and new features 15m

        During 2019 and 2020, the CERN tape archive (CTA) will receive new data from LHC experiments and import existing data from CASTOR, which will be phased out for LHC experiments before Run 3.

        This contribution will present the statuses of CTA as a service and of its integration with EOS and FTS and the data flow chains of LHC experiments.

        The latest enhancements and additions to the software as well as the development outlook will be presented. With the development of the repack function, a necessary behind-the-scenes feature, CTA can now take over custodial data and handle media migration, compaction and failures. Further metadata handling optimizations allowed doubling the maximum file rate performance to 200Hz per queue.

        New retrieve scheduling options are being developed at the request of experiments, with optional FIFO behavior to ensure better control of the timing for datasets retrieve, and fair share support for competing activities within the same VO.

        Support for multiple backend databases (Oracle, PostgreSQL, MySQL) have been developed at CERN and contributed by external institutes.

        This contribution will also report on the challenges of and solutions for migrating data from the decades old CASTOR to CTA. The practical example of the preparation for the migration of Atlas data will be presented.

        Speaker: Eric Cano (CERN)
      • 12:00
        Seeking an alternative to tape-based custodial storage 15m

        In November 2018, the KISTI Tier-1 centre started a project to design, develop and deploy a disk-based custodial storage with error rate and reliability compatible with a tape-based storage. This project has been conducted in the collaboration between KISTI and CERN, especially the initial system design was laid out from the intensive discussion with CERN IT and ALICE. The initial system design of the disk-based custodial storage accommodated high density JBOD enclosures and the erasure coding implemented in EOS, the open-source storage management developed at CERN. In order to balance among system reliability, data security and I/O performance, we investigated the possible SAS connections of JBOD enclosures to the front-end node managed by EOS and the technology constraints of interconnections in terms of throughput to deal with the large number of disks. This project targets to have a production system before the start of LHC RUN3 in 2021. This year we will procure and deploy the disk-based custodial storage with the hardware specification derived from the initial system design. In this paper we present the detailed description on the initial system design, the brief results of test equipments for the procurement, the deployment of the system and the further plan of the project.

        Speaker: Sang Un Ahn (Korea Institute of Science & Technology Information (KR))
      • 12:15
        CERN Disk Storage Services: report from last data taking, evolution and future outlook towards Exabyte-scale storage 15m

        The CERN IT Storage group operates multiple distributed storage systems to support all CERN data storage requirements: the physics data generated by LHC and non-LHC experiments; object and file storage for infrastructure services; block storage for the CERN cloud system; filesystems for general use and specialized HPC clusters; content distribution filesystem for software distribution and condition databases; and sync&share cloud storage for end-user files. The total integrated capacity of these systems exceeds 0.6 Exabyte.

        Large-scale experiment data taking has been supported by EOS and CASTOR for the last 10+ years. Particular highlights for 2018 include the special Heavy-Ion run which was the last part of the LHC Run2 Programme: the IT storage systems sustained over 10GB/s to flawlessly collect and archive more than 13 PB of data in a single month. While the tape archival continues to be handled by CASTOR, the effort to migrate the current experiment workflows to the new CERN Tape Archive system (CTA) is underway.

        Ceph infrastructure has operated for more than 5 years to provide block storage to CERN IT private OpenStack cloud, a shared filesystem (CephFS) to HPC clusters and NFS storage to replace commercial Filers. S3 service was introduced in 2018, following increased user requirements for S3-compatible object storage from physics experiments and IT use-cases.

        Since its introduction in 2013, CERNBox has become a ubiquitous cloud storage interface for all CERN user groups: physicists, engineers and administration. CERNBox provides easy access to multi-petabyte data stores from a multitude of mobile and desktop devices and all mainstream, modern operating systems (Linux, Windows, macOS, Android, iOS). CERNBox provides synchronized storage for end-user’s devices as well as easy sharing for individual users and e-groups. CERNBox has also become a storage platform to host online applications to process the data such as SWAN (Service for Web-based Analysis) as well as file editors such as Collabora Online, Only Office, Draw.IO and more. An increasing number of online applications in the Windows infrastructure uses CIFS/SMB access to CERNBox files.

        CVMFS provides software repositories for all experiments across the WLCG infrastructure and has recently been optimized to efficiently handle nightly-builds. While AFS continues to provide general-purpose filesystem for internal CERN users, especially as $HOME login area on central computing infrastructure, the migration of project and web spaces has significantly advanced.

        In this paper, we report on the experiences from the last year of LHC RUN2 data taking and evolution of our services in the past year.. We will highlight upcoming changes and future improvements and challenges.

        Speaker: Luca Mascetti (CERN)
    • 11:00 12:30
      Track 5 – Software Development: Common tools 1: GUI, geometry, analysis, data models Riverbank R2

      Riverbank R2

      Adelaide Convention Centre

      Convener: Mihaela Gheata (Institute of Space Science (RO))
      • 11:00
        New web-based ROOT GUI 15m

        RWebWindow class builds the core functionality for web-based widgets in ROOT. It combines all necessary server-side components and provides communication channels with multiple JavaScript clients.

        Following new ROOT widgets are build based on RWebWindow functionality:

        • RCanvas – ROOT7 canvas for drawing all kinds of primitives, including
          histograms and graphs
        • RBrowser – hierarchical objects and files browser with integrated viewer
        • RFitPanel – fit panel for ROOT6/ROOT7 data classes
        • REve – interactive event display
        • RGeomViewer – geometry viewer
        • TWebCanvas – lightweight web display for ROOT6 TCanvas

        OpenUI5 is used to build user interfaces like combo boxes, menus, dialogs in all web-based widgets. Special component was implemented to provide efficient and scalable hierarchy browser.

        Speaker: Serguei Linev (GSI - Helmholtzzentrum fur Schwerionenforschung GmbH (DE))
      • 11:15
        Recent developments in histogram libraries 15m

        Boost.Histogram, a header-only C++14 library that provides multi-dimensional histograms and profiles, is now available in Boost-1.70. It is extensible, fast, and uses modern C++ features. Using template meta-programming, the most efficient code path for any given configuration is automatically selected. The library includes key features designed for the particle physics community, such as optional under- and overflow bins, weighted increments, reductions, growing axes, thread-safe filling, and memory-efficient counters with high-dynamic range.

        Python bindings for Boost.Histogram are being developed in the Scikit-HEP project to provide a fast, easy-to-install package as a backend for other Python libraries and for advanced users to manipulate histograms. Versatile and efficient histogram filling, effective manipulation, multithreading support, and other features make this a powerful tool. This library has also driven package distribution efforts in Scikit-HEP, allowing binary packages hosted on PyPI to be available for a very wide variety of platforms.

        Two other libraries fill out the remainder of the Scikit-HEP Python histogramming effort. Aghast is a library designed to provide conversions between different forms of histograms, enabling interaction between histogram libraries, often without an extra copy in memory. This enables a user to make a histogram in one library and then save it in another form, such as saving a Boost.Histogram in ROOT. And Hist is a library providing friendly, analyst-targeted syntax and shortcuts for quick manipulations and fast plotting using these two libraries.

        Speaker: Henry Fredrick Schreiner (University of Cincinnati (US))
      • 11:30
        C++ Modules in ROOT and Beyond 15m

        C++ Modules come in C++20 to fix the long-standing build scalability problems in the language. They provide an io-efficient, on-disk representation capable to reduce build times and peak memory usage. ROOT employs the C++ modules technology further in the ROOT dictionary system to improve its performance and reduce the memory footprint.

        ROOT with C++ Modules was released as a technology preview in fall 2018, after intensive development during the last few years. The current state is ready for production, however, there is still room for performance optimizations. In this talk, we show the roadmap for making the technology default in ROOT. We demonstrate a global module indexing optimization which allows reducing the memory footprint dramatically for many workflows. We will report user feedback on the migration to ROOT with C++ Modules.

        Speaker: Oksana Shadura (University of Nebraska Lincoln (US))
      • 11:45
        PODIO: recent developments in the Plain Old Data EDM toolkit 15m

        PODIO is a C++ toolkit for the creation of event data models (EDMs) with a fast and efficient I/O layer, developed in the AIDA2020 project. It employs plain-old-data (POD) data structures wherever possible, while avoiding deep object-hierarchies and virtual inheritance. A lightweight layer of handle classes provides the necessary high-level interface for the physicist, such as support for inter-object relation-ships, convenient iteration through objects or automatic memory-management. PODIO creates all EDM code from simple instructive YAML files, describing the actual EDM entities.

        Since its original development PODIO has been very actively used for Future Circular Collider studies. In its original version, the underlying I/O was entirely based on the automatic streaming code generated with ROOT dictionaries. Recently two additional I/O implementations have been added. One is based on HDF5 and the other uses SIO, a simple binary I/O library provided by LCIO (the Linear Collider I/O EDM). HDF5 is heavily used in many other science fields as well as by the machine learning community. Providing the option to persistify the EDM in this way allows HEP data to be used with tools based around that ecosystem. The SIO implementation exploits the array-of-struct data layout with the goal of optimising the I/O performance.

        We briefly introduce the main features of PODIO and then report on recent developments with a focus on performance comparisons between the three available I/O implementations. We conclude with presenting recent activities on porting the well-established LCIO EDM to PODIO, thereby discussing the possibility of defining a common HEP-EDM that is shared by all future collider studies.

        Speaker: Frank-Dieter Gaede (Deutsches Elektronen-Synchrotron (DE))
      • 12:00
        Web-based ROOT geometry viewer 15m

        REve library of ROOT provides web-based event display and includes all necessary components for geometry visualization. These components are reused in the new web-based geometry viewer, where ROOT geometries of arbitrary complexity can be displayed.
        With new geometry viewer one could browse hierarchy of the geometry nodes, change individual node/volume/shape attributes, search volumes by name and highlight them on overall drawing, analyze and show volumes overlaps.
        RWebWindow class helps to implement “offline” features, when any geometry drawing can be reproduced in the web browsers without need of running C++ code. This allow sharing complex and interactive geometry drawings in the world-wide web.

        Speaker: Serguei Linev (GSI - Helmholtzzentrum fur Schwerionenforschung GmbH (DE))
      • 12:15
        mesh2gdml: Importing CAD and meshed geometries into Geant4 15m

        Geant4 is the de facto HEP standard for simulating the interaction of particles with materials and fields. The software toolkit provides a very rich library of basic geometrical shapes, often referred to as “primitives”, plus the ability to define compound geometries, making it capable of supporting extremely complex physical structures. The ability to directly import CAD geometries into Geant4 is an often requested feature, despite the recognized limitations of the difficulty in accessing proprietary formats, the mismatch between level of detail in producing a part and simulating it, the often disparate approaches to parent-child relationships and the difficulty in maintaining or assigning material definitions to parts. The main impediment to the importation of CAD files into Geant4 has been their proprietary formats. Thanks to the proliferation of rapid prototyping and additive manufacturing processes, the surface tesselation language (STL) format is the industrial standard for handling triangulated meshes and is ubiquitous as an export format for both CAD and other 3D modelling software. Geant4 fully and natively supports an xml-based Geometry Description Markup Language (GDML) which supports tesselated volumes to define geometries. By targeting meshed geometries instead of specific CAD formats, this approach also provides a useful solution in cases where the objects are intrinsically irregular, such as biological phantoms. The STL format consists of a plain list of three-dimensional corner point coordinates (vertex) and flat triangles (facet) with an associated normal vector, making it an ideal candidate for importation of volumes into Geant4 as G4TessellatedSolids. Since there is no other structure in an STL file, one has to also solve the problem of creating "topology from a bucket of facets", which we have done. The one area requiring manual intervention is the assignment of material to the newly created solid or solids. We discuss a few pathways forward to solve this issue, including the use of a graphical user interface. Despite the inherent performance issues related to navigating through geometries composed of many individual facets and the requirement that material be assigned manually to volumes during the translation process, we believe the approach outlined in this talk provides access to a wider range of geometry inputs and will prove to be useful to a number of user communities from disparate fields. In this talk, we present the current status of mesh2gdml, a solution which can be used to convert an STL file into a collection of G4TessellatedSolids which can be imported directly into Geant4 via GDML.

        Speaker: Norman Anthony Graf (SLAC National Accelerator Laboratory (US))
    • 11:00 12:30
      Track 6 – Physics Analysis: Pheno fits / Analysis preservation Hall G

      Hall G

      Adelaide Convention Centre

      Convener: Martin White (University of Adelaide (AU))
      • 11:00
        Constraining effective field theories with machine learning 15m

        An important part of the LHC legacy will be precise limits on indirect effects of new physics, framed for instance in terms of an effective field theory. These measurements often involve many theory parameters and observables, which makes them challenging for traditional analysis methods. We discuss the underlying problem of “likelihood-free” inference and present powerful new analysis techniques that combine physics insights, statistical methods, and the power of machine learning. We have developed MadMiner, a new Python package that makes it straightforward to apply these techniques. In example LHC problems we show that the new approach lets us put stronger constraints on theory parameters than established methods, demonstrating its potential to improve the new physics reach of the LHC legacy measurements. While we present techniques optimized for particle physics, the likelihood-free inference formulation is much more general, and these ideas are part of a broader movement that is changing scientific inference in fields as diverse as cosmology, genetics, and epidemiology.

        Speaker: Alexander Held (University of British Columbia (CA))
      • 11:15
        Monte Carlo methods for global QCD analysis of parton distributions 15m

        Extracting information about the quark and gluon (or parton) structure of the nucleon from high-energy scattering data is a classic example of the inverse problem: the experimental cross sections are given by convolutions of the parton probability distributions with process-dependent hard coefficients that are perturbatively calculable from QCD. While most analyses in the past have been based on the maximum likelihood approach, it is becoming evident that Bayesian likelihood methods using Monte Carlo (MC) sampling techniques are increasingly needed to thoroughly explore the parameter space associated with the quantum probability distributions. This talk will review the recent developments in the application of MC techniques to global QCD analyses, which are leading to a paradigm shift in the phenomenological study of partonic structure of the nucleon, including the first simultaneous global analysis of collinear parton distributions and parton to hadron fragmentation functions. Future applications to the study of the 3-dimensional structure of the nucleon, in terms of transverse momentum dependent parton distributions, will be outlined.

        Speaker: Dr Wally Melnitchouk (Jefferson Lab)
      • 11:30
        PhaseTracer: finding cosmological phases and calculating transition properties 15m

        Phase transitions played an important role in the very early evolution of the Universe. We present a C++ software package (PhaseTracer) for finding cosmological phases and calculating transition properties involving single or multiple scalar fields. The package first maps the phase structure by tracing the vacuum expectation value (VEV) of the potential at different temperatures, then finds out all possible transitions between them, and finally computes their critical temperatures and transition strengths. PhaseTracer is constructed with modularity, flexibility and practicality in mind. It is fast and stable, can receive potential provided by other packages and connect to BubbleProfiler to calculate action of transition.

        Speaker: Yang Zhang
      • 11:45
        The CMS approach to analysis preservation 15m

        The CERN analysis preservation portal (CAP) comprises a set of tools and services aiming to assist researchers in describing and preserving all the components of a physics analysis such as data, software and computing environment. Together with the associated documentation, all these assets are kept in one place so that the analysis can be fully or partially reused even several years after the publication of the original scientific results.
        An experiment-specific submission and retrieval interface has been developed for the CMS collaboration.
        It integrates with the CMS internal analysis registry (CADI) to capture all analyses with basic information, complemented with a detailed submission form for full information. The CMS data aggregation system (DAS) is interfaced to the deposit form to assist in filling in exact dataset names used in the analysis to ensure searchability.
        Efforts are ongoing to describe physics content for an intelligent retrieval, and to interface with container solutions for full reproducibility for selected test cases.

        Speaker: Lara Lloret Iglesias (CSIC - Consejo Sup. de Investig. Cientif. (ES))
      • 12:00
        Likelihood preservation and statistical reproduction of searches for new physics 15m

        Likelihoods associated with statistical fits in searches for new physics are beginning to be published by LHC experiments on HEPData [arXiv:1704.05473]. The first of these is the search for bottom-squark pair production by ATLAS [ATLAS-CONF-2019-011]. These likelihoods adhere to a specification first defined by the HistFactory p.d.f. template [CERN-OPEN-2012-016]. This is per-se independent of its implementation in ROOT and it is useful to be able to run statistical analysis outside of the ROOT and RooStats/RooFit framework. We introduce a JSON schema that fully describes the HistFactory statistical model and is sufficient to reproduce key results from published ATLAS analyses. Using two independent implementations of the model, one in ROOT and one in pure Python, we reproduce the sbottom multi-$b$ limits using the published likelihoods on HEPData underscoring the implementation independence and long-term viability of the archived data.

        Speaker: Matthew Feickert (Southern Methodist University (US))
      • 12:15
        CERN analysis preservation and reuse framework: FAIR research data services for LHC experiments 15m

        In this paper we present the CERN Analysis Preservation service as a FAIR (Findable, Accessible, Interoperable and Reusable) research data preservation repository platform for LHC experiments. The CERN Analysis Preservation repository allows LHC collaborations to deposit and share the structured information about analyses as well as to capture the individual data assets associated to the analysis. We describe the typical data ingestion pipelines how an individual physicist can preserve and share their final n-tuples, ROOT macros, Jupyter notebooks or even full analysis workflow code and any intermediate datasets of interest for preservation within the restricted context of experimental collaboration. We discuss the importance of annotating the deposited content with high-level structured information about physics concepts in order to promote information discovery and knowledge sharing inside the collaboration. Finally, we describe techniques taken to facilitate the reusability of preserved data assets by capturing and re-executing reproducible recipes and computational workflows using the REANA Reusable Analysis platform.

        Speaker: Pamfilos Fokianos (CERN)
    • 11:00 12:30
      Track 7 – Facilities, Clouds and Containers: Non-LHC experiments Riverbank R7

      Riverbank R7

      Adelaide Convention Centre

      Convener: Christoph Wissing (Deutsches Elektronen-Synchrotron (DE))
      • 11:00
        Evolution of the LHAASO Distributed Computing System based Cloud 15m

        The LHAASO(Large High Altitude Air Shower Observatory) experiment of IHEP is located in Daocheng, Sichuan province (at the altitude of 4410 m). The main scientific goals of LHAASO are searching for galactic cosmic ray origins by extensive spectroscopy investigations of gamma ray sources above 30TeV. To accomplish these goals, LHAASO contains four detector arrays, which generates huge amounts of data and requires mass storage and high performance computing system. And the dedicated computing resource of LHAASO locates in Beijing, Daocheng and Chengdu as well as resources from collaborated organizations. How to establish a distributed computing system making the distributed resources work together and provide a good computing service for LHAASO is very important and urgent. However, it faces high operation and maintenance costs, system instability and other issues especially from remote sites.
        In this paper we will describe the evolution of LHAASO distributed computing system based on virtualization and cloud computing technologies. Particularly, we discuss the key points of integrating distributed resources. A solution of integrating cross-domain resources is proposed, which adopt the Openstack+HTCondor to make the distributed resource work as a whole resource pool. A flexible resource scheduling strategy and a job scheduling policy are presented to realize the resource expansion on demand and the efficient job scheduling to remote sites transparently, so as to improve the overall resource utilization. We will also introduce the deployment of the computing system located in Daocheng, the LHAASO observation base using cloud-based architecture (Openstack+Kubernetes), which greatly helps to reduce the operation and maintenance cost as well as to make sure the system availability and stability. Finally, how to monitor the distributed computing system will be illustrated.

        Speaker: Dr Qiulan Huang (Institute of High Energy of Physics,CAS)
      • 11:15
        VIRGO and Gravitational Waves computing in Europe 15m

        VIRGO is an interferometer for the detection of Gravitational Waves at the European Gravitational Observatory in Italy. Along with the two LIGO interferometers in the US, VIRGO is being used to collect data from astrophysical sources such as compact binary coalescences, and is currently running its third observational period, collecting gravitational wave events at a rate if more than one per week.

        Data from the interferometer are processed by running search pipelines for a number of expected signals, from coalescing compact binaries to continuous waves and burst events. Furthermore, detector characterisation studies are run. Some of the processing needs to be done with low latency, in order to be able to provide triggers for other observatories and make multi-messenger observations possible. Deep searches are run offline on external computing centres. Thus, data needs also to be reliably and promptly distributed from the EGO site to computer centres in Europe and the US for further analysis and archival storage.

        Two of the defining characteristics of VIRGO computing are the heterogeneity of the activities and the need to interoperate with LIGO. A very wide array of analysis pipelines differing in scientific target, implementation details and running environment assumptions have to be allowed to run ubiquitously and uniformly on dedicated resources and, in perspective, on heterogeneous infrastructures.

        The current status, possible strategies and outlook are discussed.

        Speaker: Dr Stefano Bagnasco (Istituto Nazionale di Fisica Nucleare, Torino)
      • 11:30
        online computing for new generation photon science experiments 15m

        Experiments in Photon Science at DESY will, in future, undergo significant changes in terms of data volumes, data rates and most important, to fully enable online (synchronous to experiment) data analysis. Primary goal is to support new type of experimental setups requiring significant computing effort to perform controlling and data quality monitoring, allow effective data reductions and, finally, allowing discarding of raw data before storing at all. Different to pure HEP experiments, the software infrastructure has to be standardized and easily (esp. fast) usable by many experiments with different physics goals. We will discuss the plans and work done regarding the hardware infrastructure, the system level middleware (i.e. container, storage connection, networks) and the higher level middleware covering low latency data access and selection (including metadata queries) direct connection the DAQ at detector level to the data processing code (mostly developed by experimentators and running on multiple nodes in parallel. Finally we will show the most current status and first results from initial testing with 'artificial' and real experimental setups.

        Speaker: Mr Martin Gasthuber (DESY)
      • 11:45
        High-availability on-site deployment to heterogeneous architectures for Project 8 and ADMX 15m

        Project 8 is applying a novel spectroscopy technique to make a precision measurement of the tritium beta-decay spectrum, resulting in either a measurement of or further constraint on the effective mass of the electron antineutrino. ADMX is operating an axion haloscope to scan the mass-coupling parameter space in search of dark matter axions. Both collaborations are executing medium-scale experiments, where stable operations last for three to nine months and the same system is used for development and testing between periods of operation. It is also increasingly common to use low-cost computing elements, such as the Raspberry Pi, to integrate computing and control with custom instrumentation and hardware. This leads to situations where it is necessary to support software deployment to heterogeneous architectures on rapid development cycles while maintaining high availability. Here we present the use of docker containers to standardize packaging and execution of control software for both experiments and the use of kubernetes for management and monitoring of container deployment in an active research and development environment. We also discuss the advantages over more traditional approaches employed by experiments at this scale, such as detached user execution or custom control shell scripts.

        Speaker: Benjamin LaRoque
      • 12:00
        IceCube Real-time Processing in AWS 15m

        IceCube sends out real-time alerts for neutrino events to other multi-messenger observatories around the world, including LIGO/VIRGO and electromagnetic observatories. The typical case is to send out an initial alert within one minute, then run more expensive processing to refine the direction and energy estimates and send a follow-on message. This second message has averaged 40 to 60 minutes of delay over the last two years, but is essential for pointing telescopes that have a small field of view. Recently we have moved from a local compute cluster to bursting in Amazon’s cloud to increase parallelization and decrease the follow-on message delay. This involved creating new distribution architectures for event data and collection of results, to scale beyond a few thousand workers. We will give an overview of the steps we took, how many machines we are able to scale to, and the improvement in follow-on message delay.

        Speakers: Benedikt Riedel (University of Wisconsin-Madison), Benedikt Riedel (University of Wisconsin-Madison)
      • 12:15
        Designing a Federated Regional Centre for SKA Computing 15m

        This paper describes the work done by the AENEAS project to develop a concept and design for a distributed, federated, European SKA Regional Centre (ESRC) to support the compute, storage, and networking that will be required to achieve the scientific goals of the Square Kilometre Array (SKA).
        The AENEAS (Advanced European Network of E-infrastructures for Astronomy with the SKA) project is a 3 year initiative funded by the Horizon 2020 program of the European Commission which started in 2017 and includes European, EC e-infrastructure, and Global partners. SKA is similar to LHC and Particle Physics in being a global collaboration of scientists and requiring world wide distributed computing infrastructure to extract the science from the data products produced by the telescopes; however the organisation of the SKA data and the computing required is quite different.
        The context will be set by briefly describing the data rates and types produced by the two telescopes that comprise the SKA, followed by an outline of the high-level requirements for the ESDC. Estimates of the scale of the computing and storage provide insight into how one might form a federated distributed compute and storage facility.
        The talk will cover the different data placement models that AENEAS has created, in collaboration with the SKA community, and how they influence the design and cost of the global data transfer and access networks that will support SKA.
        The talk will show how the computing and storage requirements for the ESRC have been estimated based on use cases and scenarios related to the different types of data products that SKA telescopes will produce. It will also cover how middleware infrastructure such as RUCIO might be used.
        The design aspects required to ensure user access to and interaction with the computing infrastructure and SKA data to provide SKA science outputs will be covered. A seamless Authentication and Authorization Infrastructure (AAI) and other federated services are required also for global interoperability.
        A set of data challenges have been undertaken by the SKA community and some of these will be presented to conclud the talk.

        Speaker: Dr Richard Hughes-Jones (GEANT Association)
    • 11:00 12:30
      Track 8 – Collaboration, Education, Training and Outreach: Collaboration and training Riverbank R4

      Riverbank R4

      Adelaide Convention Centre

      Convener: Ilya Komarov
      • 11:00
        User documentation and training at Belle II 15m

        Belle II is a rapidly growing collaboration with members from
        113 institutes spread around the globe. The software development team of
        the experiment, as well as the software users, are very much
        decentralised. Together with the active development of the software,
        such decentralisation makes the adoption of the latest software
        releases by users an essential, but quite challenging task.
        To ensure an up-to-date state of the documentation, we adopted the
        policy of in-code documentation and configured a website that allows to
        get the documentation for specific release. To prevent tutorials becoming
        outdated, they are covered by unit-tests. For the user support, we use
        a question and answer service that not only allows to reduce repetition
        of the same questions, but also turned out to be a place for discussions
        among the experts. A prototype of a meta search engine for the different
        sources of documentation has been developed. For training of the new
        users, we organize centralised StarterKit workshops attached to the
        collaboration meetings. The materials of the workshop are available
        through a jupyterhub server and used in local training sessions.

        Speaker: Ilya Komarov
      • 11:15
        An Institute for Scientific Software, connecting Applied Computing and Data Intensive Sciences 15m

        With the ever increasing size of scientific collaborations and complexity of scientific instruments the software needed to acquire, process and analyze the gathered data is gaining in complexity and size too. Unfortunately the role and career path of scientists and engineers working on software R&D and developing scientific software is neither clearly established nor defined in many fields of natural science. In addition the exchange of information between scientific software development and computer science departments at universities or computing schools is scattered and de-fragmented into individual initiatives. To address the above issues we propose a new effort on an European level, which concentrates on strengthening the role of software developers in natural sciences, acts as a hub for exchange of ideas among different stakeholders in computer science and scientific software and forms a lobbying forum for software engineering in natural sciences on an international level. This contribution discusses in detail the motivation, role and interplay with other initiatives of a "Software Institute for Data Intensive Science" which is currently being discussed between research institutes, universities and funding agencies in Europe. In addition to the current status, an outlook on future prospects of this initiative will be given.

        Speaker: Stefan Roiser (CERN)
      • 11:30
        Enabling HEP software development for the 2020s - a coherent vision for training 15m

        Developing, maintaining, and evolving the algorithms and
        software implementations for HEP experiments will continue for many
        decades. In particular, the HL-LHC will start collecting data 8 or
        9 years from now, and then acquire data for at least another decade.
        Building the necessary software requires a workforce with a mix of
        HEP domain knowledge, advanced software skills, and strong connections
        to other related disciplines. The investments to grow this workforce
        must begin today. Several new efforts are working towards this goal, like
        the IRIS-HEP software institute, the FIRST-HEP project and the HEP Software
        Foundation. Past software training practices have largely been set of
        individual and disconnected activities. The HEP community planning process
        has established a coherent vision for software training as part of a larger
        framework where researchers progress from acquiring basic skills training
        through user training in existing software to training in skills needed to
        development new research software. This presentation will articulate this
        vision and describe recent efforts to enhance the sustainability, re-usability
        and impact of the software training activities as well as related education
        and outreach programs.

        Speaker: Peter Elmer (Princeton University (US))
      • 11:45
        Email-based threats - addressing the human factor 15m

        Since years, e-mail is one of the main attack vectors that organisations and individuals face. Malicious actors use e-mail messages to run phishing attacks, to distribute malware, and to send around various types of scams. While technical solutions exist to filter out most of such messages, no mechanism can guarantee 100% efficiency. Recipients themselves are the next, crucial layer of protection - but unfortunately, they fall for the various tricks used by attackers way too often.

        This presentation will start with a quick overview of social engineering tricks used these days in both generic and targeted attacks. We will briefly look at strengths and limitations of technical counter-measures such as spam filters and antimalware protection. We will then focus on the human aspect, and in particular on user education. Lessons from awareness raising campaigns run at CERN in the last few years will be discussed, and compared to approaches employed by other organisations, and proposed by companies offering commercial solutions.

        Speaker: Sebastian Lopienski (CERN)
      • 12:00
        New developments for ALICE MasterClasses and the new Particle Therapy MasterClass 15m

        Heavy-ion physics is present within the "International MasterClasses" for almost ten years. New developments aiming at expanding their scope and reach are presented in this talk.

        First, in line with the physics research of typical heavy-ion experiments, three measurements were developed based on actual data analysis in the ALICE experiment at CERN/LHC. They correspond to the most important observables studied to characterise the properties of matter produced in energetic collisions of ions or protons. Then, with the aim to highlight applications from fundamental research for society a new MasterClass was developed on Particle Therapy for cancer treatment highlighting the benefits of therapy with carbon ions. A good example is the fact that for this therapy the heavy-ion research centre GSI, Germany, played a pioneering role in Europe, in 90s, leading to the construction of the HIT therapy centre in Heidelberg. Both masterclasses had challenging requirements on software developments as they had to be realistic but also easy to use.

        In particular, recent developments resulted in a single modular framework integrating all three ALICE measurements supporting coherent further expansions. This framework can be built as a standalone application which can run on Linux by a simple double click, without the need of separate ROOT installation while necessary ROOT libraries are built in the executable using the AppImage format. The work of migrating the framework to a web environment has started and a proof of concept is presented.

        For the Particle Therapy Masterclass (PTMC) the software used, matRad based on Matlab, is a professional open source toolkit developed by the cancer therapy research centre DKFZ in Heidelberg for research and training. It allows optimising the dose to be delivered to the patient (treatment planning) using photons, protons or ions. For the masterclass, first a phantom is used to explain the concepts and it is followed by treatments of head and liver tumours. PTMC demonstrates applications of fundamental physics research for cancer treatment, namely, the use of accelerators, detectors, and understanding interactions of different particles with matter.

        Speaker: Lukasz Kamil Graczykowski (Warsaw University of Technology (PL))
    • 11:00 12:30
      Track 9 – Exascale Science: Scheduling, computing environment Riverbank R1

      Riverbank R1

      Adelaide Convention Centre

      Convener: Fabio Hernandez (IN2P3 / CNRS computing centre)
      • 11:00
        Exploiting network restricted compute resources with HTCondor: a CMS experiment experience 15m

        In view of the increasing computing needs for the HL-LHC era, the LHC experiments are exploring new ways to access, integrate and use non-Grid compute resources. Accessing and making efficient use of Cloud and supercomputer (HPC) resources present a diversity of challenges. In particular, network limitations from the compute nodes in HPC centers impede CMS experiment pilot jobs to connect to its central HTCondor pool to receive the actual payload jobs to be executed. To cope with this limitation, new features have been developed in both HTCondor and the CMS resource scheduling and workload management infrastructure. In this novel approach, a bridge is set up outside the HPC center and the communications between HTCondor daemons are relayed through a shared file system. We have used this strategy to exploit the Barcelona Supercomputing Center (BSC) resources, the main Spanish HPC site. CMS payloads are claimed by HTCondor startd daemons running at the nearby PIC Tier-1 center and routed to BSC compute nodes through the bridge. This fully enables the connectivity of CMS HTCondor-based central infrastructure to BSC resources via PIC HTCondor pool. Other challenges have included building custom Singularity images with CMS software releases, bringing conditions data to payload jobs, and custom data handling between BSC and PIC. This contribution describes this technical prototype, its deployment, the functionality and scalability tests performed, along with the results obtained when exploiting the BSC resources using these novel approaches. A key aspect of the technique described in this contribution is that it could be universally employed in similar HPC environments elsewhere.

        Speaker: Jose Flix Molina (Centro de Investigaciones Energéti cas Medioambientales y Tecno)
      • 11:15
        Improvements in utilisation of the Czech national HPC center by ATLAS distributed computing 15m

        ATLAS distributed computing is allowed to opportunistically use resources of the Czech national HPC center IT4Innovations in Ostrava. The jobs are submitted via an ARC Compute Element (ARC-CE) installed at the grid site in Prague. Scripts and input files are shared between the ARC-CE and the shared file system located at the HPC, via sshfs. This basic submission system has worked there since the end of 2017.
        Several improvements were made to increase the amount of resources that ATLAS can use. The most significant improvement was the migration of the submission system to enable pre-emptable jobs, as the HPC center management made the decision to use pre-emption on opportunistic jobs. Another improvement of the submission system is related to the sshfs connection, which seemed to be a limiting factor of the system. Now, the submission system consists of several ARC-CE machines. Also, various parameters of sshfs were tested in an attempt to increase throughput. As a result of the improvements, the utilisation of the Czech national HPC center by the ATLAS distributed computing increased.

        Speaker: Michal Svatos (Acad. of Sciences of the Czech Rep. (CZ))
      • 11:30
        Large scale fine grain simulation workflows ("Jumbo Jobs") on HPC's by the ATLAS experiment 15m

        The ATLAS experiment is using large High Performance Computers (HPC's) and fine grained simulation workflows (Event Service) to produce fully simulated events in an efficient manner. ATLAS has developed a new software component (Harvester) which provides resource provisioning and workload shaping. In order to run effectively on the largest HPC machines, ATLAS develop Yoda-Droid software to orchestrate the MPI communication between Harvester and the simulation payload running on over 1000 nodes simultaneously. In this way over 130,000 cores can simultaneously produce simulated Monte Carlo events for ATLAS. The PanDA system also had to be changed to produce "jumbo jobs" capable of simulated over 1 Million events per submission to the local HPC scheduling systems.
        This presentation will describe in detail the changes to PanDA to enable jumbo jobs and the Yoda-Droid software. Scaling and efficiency measurements will be presented. Results from deployment, integration and operation of the new software at the Titan, Cori and Theta HPC machines will be shown.

        Speaker: Doug Benjamin (Argonne National Laboratory (US))
      • 11:45
        Physics Data Production on HPC: Experience to be efficiently running at scale 15m

        The Solenoidal Tracker at RHIC (STAR) is a multi-national supported experiment located at Brookhaven National Lab and is currently the only remaining running experiment at RHIC. The raw physics data captured from the detector is on the order of tens of PBytes per data acquisition campaign, which makes STAR fit well within the definition of a big data science experiment. The production of the data has typically run using an High Throughput Computing (HTC) approach either done on a local farm or via Grid computing resources. Especially, all embedding simulations (complex workflow mixing real and simulated events) have been run on standard Linux resources at NERSC/PDSF. However, as per April 2019 PDSF has been retired and High Performance Computing (HPC) resources such as the Cray XC-40 Supercomputer known as “Cori” have become available for STAR’s data production as well as embedding. STAR has been the very first experiment to show feasibility of running a sustainable data production campaign on this computing resource. In this contribution, we hope to share with the community the best practices for using such resource efficiently.

        The use of Docker containers with Shifter is the standard approach to run on HPC at NERSC – this approach encapsulates the environment in which a standard STAR workflow runs. From the deployment of a tailored Scientific Linux environment (with the set of libraries and special configurations required for STAR to run) to the deployment of third-party software and the STAR specific software stack, we’ve learned it has become impractical to rely on a set of containers containing each specific software release. To this extent, a solution based on CVMFS for the deployment of software and services has been deployed but it doesn’t stop there. One needs to make careful scalability considerations when using a resource like Cori, such as avoiding metadata lookups, scalability of distributed filesystems, and real limitations of containerized environments on HPC. Additionally, CVMFS clients are not compatible on Cori nodes and one needs to rely on an indirect NFS mount scheme using special DVS servers designed to forward data to worker nodes. In our contribution, we will discuss our strategies from the past and our current solution based on CVMFS. The second focus of our presentation will be to discuss strategies to find the most efficient use of database Shifter containers serving our data production (a near “database as a service” approach) and the best methods to test and scale your workflow efficiently.

        Speaker: Michael Poat (Brookhaven National Laboratory)
      • 12:00
        Advancing physics simulation and analysis workflows from customized local clusters to Cori - the HPC optimized sub-million cores system at NERSC 15m

        Abstract: Over the last few years, many physics experiments migrated their computations from customized locally managed computing clusters to orders of magnitude larger multi-tenant HPC systems often optimized for highly parallelizable long-runtime computations. Historically, physics simulations and analysis workflows were designed for a single core CPUs with abundant RAM, plenty of local storage, direct control of the software stack and job scheduler, exclusive access to physically localized hardware, and predictable steady throughput. We will discuss what changes needed to happen in terms of the data pipeline organization, software, and user habits when computations are executed at scale on Cori, where none of those assumptions are true anymore.

        STAR experiment at BNL took on the challenge as one of the first. We will discuss the efficient solutions for sustainable processing of experimental data at HPC system 5000 miles away from an experiment, with 2-way just-in-time data transfer. Due to limited administrative privileges at HPC machines, Docker/Shifter become one of the main vehicles to transport custom vetted code to the HPC environment, supplemented by CVMFS software delivery system mounted via DVS servers providing local cache. DayaBay, LZ, ATLAS, and Majorana experiments followed this journey of transformations, by developing schemes for injecting short leaving single core tasks to multi-node, 1000s-core, long run-time jobs scheduled on Cori - the most efficient way to compete with other tenants for CPU cycles. The high variability of scheduling required assembly of ‘convoy’ jobs composed of many nodes dedicated to the execution of tasks and designating one node to carry a read-only clone of the database. The computing capability of one ‘convoy’ can be compared to the whole PDSF and one user can schedule 10s of such jobs to run concurrently on Cori. The fine-tuning of tasks concurrency per node to maximize the output for per-node charge-hour given a rather low RAM/CPU ratio and benefiting from 2- or 4-threading capability will be also discussed.

        Speaker: Jan Balewski (Lawrence Berkeley National Lab. (US))
    • 12:30 14:00
      Lunch 1h 30m
    • 14:00 15:30
      Track 1 – Online and Real-time Computing: Future upgrades Riverbank R5

      Riverbank R5

      Adelaide Convention Centre

      Convener: Jennifer Ngadiuba (CERN)
      • 14:00
        Mass storage interface LTSM for FAIR Phase 0 data acquisition 15m

        Since 2018 several FAIR Phase 0 beamtimes have been operated at GSI, Darmstadt. Here the new challenging technologies for the upcoming FAIR facility shall be tested while various physics experiments are performed with the existing GSI accelerators. One of these challenges concerns the performance, reliability, and scalability of the experiment data storage. A new system for archiving the data from the GSI and FAIR Phase 0 experiments to long-term storage has been developed in the last years and was successfully employed in the 2019 spring beamtime.
        Raw data as collected by event building software of large scale detector data acquisition has to be safely written to a mass storage system like a magnetic tape library. Besides this long term archive, it is often required to process this data as soon as possible on a high performance compute farm. A full near-online analysis is essential to give a fast feedback on the quality of the data acquired, allowing to improve detector and measurement parameters such as alignment, calibration, high voltage and electronics set-up, etc.
        The C library LTSM ("Lightweight Tivoli Storage Management") has been developed at GSI IT department based on the IBM TSM software. It provides a file API that allows to write raw listmode data files via TCP/IP sockets directly to an IBM TSM storage server. Moreover, the LTSM library offers Lustre HSM ("Hierarchical Storage Management") capabilities for seamlessly archiving and retrieving data stored on Lustre file system and TSM server. That is, data can be automatically archived to low-cost storage media such as magnetic tapes and seamlessly retrieved when accessing the data on the Lustre file system.
        In spring 2019 LTSM has been employed at the FAIR Phase 0 beamtimes at GSI. For the HADES experiment LTSM was implemented into the DABC ("Data Acquisition Backbone Core") event building software. During the 4 weeks of Ag+Ag@1.58 AGeV beam, the HADES event builders have transferred about 400 TB data via 8 parallel 10 GbE sockets, both to the TSM archive and to the "GSI green cube" HPC farm. For other FAIR Phase 0 experiments using the vintage MBS ("Multi Branch System") event builders, an LTSM gateway application has been developed to connect the legacy RFIO ("Remote File I/O") protocol of these DAQ systems with the new storage interface.
        In this contribution the novel LTSM mass storage architecture and its applications for experiment data acquisition will be described. Experiences and storage performance measurements from HADES at FAIR Phase 0 campaign will be deeply discussed. Most recent further developments and an outlook on future enhancements will be presented.

        Speaker: Dr Joern Adamczewski-Musch (GSI Helmholtzzentrum f. Schwerionenforschung GmbH)
      • 14:15
        ATLAS Trigger and Data Acquisition Upgrades for the High Luminosity LHC 15m

        The ATLAS experiment at CERN has started the construction of upgrades
        for the "High Luminosity LHC", with collisions due to start in
        2026. In order to deliver an order of magnitude more data than
        previous LHC runs, 14 TeV protons will collide with an instantaneous
        luminosity of up to 7.5 x 10e34 cm^-2s^-1, resulting in much higher pileup and
        data rates than the current experiment was designed to handle. While
        this is essential to realise the physics programme, it presents a huge
        challenge for the detector, trigger, data acquisition and computing.
        The detector upgrades themselves also present new requirements and
        opportunities for the trigger and data acquisition system.

        The approved baseline design of the TDAQ upgrade comprises: a
        hardware-based low-latency real-time Trigger operating at 40 MHz, Data
        Acquisition which combines custom readout with commodity hardware and
        networking to deal with 5.2 TB/s input, and an Event Filter running at
        1 MHz which combines offline-like algorithms on a large commodity
        compute service augmented by hardware tracking. Commodity servers and
        networks are used as far as possible, with custom ATCA boards, high
        speed links and powerful FPGAs deployed in the low-latency parts of
        the system. Offline-style clustering and jet-finding in FPGAs, and
        track reconstruction with Associative Memory ASICs and FPGAs are
        designed to combat pileup in the Trigger and Event Filter
        respectively.

        This paper will report recent progress on the design, technology and
        construction of the system. The physics motivation and expected
        performance will be shown for key physics processes.

        Speaker: Revital Kopeliansky (Indiana University (US))
      • 14:30
        L0TP+: the Upgrade of the NA62 Level-0 Trigger Processor 15m

        The L0TP+ initiative is aimed at the upgrade of the FPGA-based Level-0 Trigger Processor (L0TP) of the NA62 experiment at CERN for the post-LS2 data taking, which is expected to happen at 100% of nominal beam intensity. Although tests performed at the end of 2018 showed a substantial robustness of the L0TP system also at full beam intensity, just hinting at a firmware fix, there are several reasons to motivate such an upgrade: i) avoid FPGA platform obsolescence, ii) make room for improvements in the firmware design leveraging a more capable FPGA device, iii) add new functionalities, iv) support the ✕4 beam intensity increase foreseen in future experiment upgrades. We singled out the Xilinx Virtex UltraScale+ VCU118 development board as the ideal platform for the project.
        L0TP+ seamless integration in the current NA62 TDAQ system and exact matching of L0TP functionalities represent the main requirements and focus of the project; nevertheless, the final design will include additional features, such as a PCIe RDMA engine to enable processing on CPU and GPU accelerators, and the partial reconfiguration of trigger firmware starting from a high level language description (C/C++). The latter capability is enabled by modern High Level Synthesis (HLS) tools, but to what extent this methodology can be applied to perform complex tasks in the L0 trigger, with its stringent latency requirements and the limits imposed by single FPGA resources, is currently being investigated. Besides, the chosen platform supports the Virtex Ultrascale+ FPGA wide I/O capabilities, allowing for straightforward integration of primitive streams from additional subdetectors in order to improve the efficiency of the trigger. As a test case for this scenario we considered the online reconstruction of the RICH detector rings on a HLS generated module, using a dedicated primitives data stream with PM hits IDs.

        Speaker: Piero Vicini (Sapienza Universita e INFN, Roma I (IT))
      • 14:45
        New Jet Feature Extraction and Topological Processor modules for ATLAS Phase-I Upgrade: from design to commissioning 15m

        To cope with the enhanced luminosity at the Large Hadron Collider (LHC) in 2021, the ATLAS collaboration is planning a major detector upgrade to be installed during the Long shutdown 2 (LS2). As a part of this, the Level 1 trigger, based on calorimeter data, will be upgraded to exploit the fine granularity readout using a new system of Feature EXtractors (FEX) and a new Topological Processor (L1Topo) that will process the Trigger Objects (TOBs) sent by FEXs and L1Muon, selecting interesting physics events by applying kinematic and angular requirements on muons, electromagnetic clusters, jets and total energy.

        The jet Feature Extractor (jFEX) has been conceived to identify small/large area jets, large-area tau leptons, missing transverse energy and the total sum of the transverse energy. It will exploit and process the data with a granularity of 0.1 x 0.1 in (eta,phi) sent at 11.2 Gb/s from the calorimeters and sort into TOBs using dedicated algorithms with a VHDL description. The TOBs will be transmitted then at 12.8 Gb/s to the L1Topo via optical links.

        The L1Topo system has been initially introduced in Run 2 to improve the trigger performance by correlating trigger objects, such as electromagnetic clusters, jets and muons, and global quantities. The system performs topological calculations on the first trigger stage, for example invariant or transverse mass cuts, angular cuts or jet energy sums. During LS2, upgraded L1Topo modules will be installed benefiting from a larger processing power available for the implemented algorithms. The jFEX and L1Topo modules exploit the latest generation of the Xilinx Ultrascale+ FPGA, XCVU9P-2FLGA2577E, characterized by large input bandwidth, up to ~ 3Tb/s per module, and large processing power.

        This contribution focuses on the design characteristics of L1Topo and jFEX and the test results performed on the L1Topo and jFEX prototypes. Both systems will be produced by Autumn 2019 to allow for completion of installation and commissioning before LHC restarts in March 2021.

        Speaker: Christian Kahra (Johannes Gutenberg Universitaet Mainz (DE))
      • 15:00
        The CMS Trigger upgrade for the HL-LHC 15m

        The CMS experiment has been designed with a two-level trigger system: the Level 1 Trigger, implemented on custom-designed electronics, and the High Level Trigger, a streamlined version of the CMS offline reconstruction software running on a computer farm. During its “Phase 2” the LHC will reach a luminosity of 7×10³⁴ cm⁻²s⁻¹ with a pileup of 200 collisions, integrating over 3000 fb⁻¹ over the full experimental run. To fully exploit the higher luminosity, the CMS experiment will introduce a more advanced Level 1 Trigger and increase the full readout rate from 100 kHz to 750 kHz. CMS is designing an efficient data-processing hardware trigger (Level-1) that will include tracking information and high-granularity calorimeter information. The current conceptual system design is expected to take full advantage of advances in FPGA and link technologies over the coming years, providing a high-performance, low-latency computing platform for large throughput and sophisticated data correlation across diverse sources. The higher luminosity, event complexity and input rate present an unprecedented challenge to the High Level Trigger, that aims to achieve a similar efficiency and rejection factor as today despite the higher pileup and more pure preselection. In this presentation we will discuss the ongoing studies and prospects for the online reconstruction and selection algorithms for the high-luminosity era.

        Speaker: Thiago Tomei Fernandez (UNESP - Universidade Estadual Paulista (BR))
      • 15:15
        The CMS Data Acquisition System for the Phase-2 Upgrade 15m

        The upgraded High Luminosity LHC, after the third Long Shutdown (LS3) will provide an instantaneous luminosity of 7.5 1034 cm-2 s-1 (levelled), with a pileup of up to 200 interactions per bunch crossing. During LS3, the CMS Detector will undergo a major upgrade to prepare for the Phase-2 of the LHC physics program, starting around 2026. The upgraded CMS detector will be read out at an unprecedented data rate of up to 50 Tb/s with an event rate of 750 kHz, selected by the level-1 hardware trigger, and an average event size of 7.5 MB. Complete events will be analysed by the High Level Trigger (HLT) using software algorithms running on standard processing nodes, and selected events will be stored permanently at a rate of up to 7.5 kHz for offline processing and analysis. Tis paper will present, the baseline design of the DAQ and HLT systems for Phase-2, taking into account the projected evolution of high speed network fabrics for event building and distribution, and the anticipated performance of general purpose CPU.

        A DAQ and Timing Hub (DTH) board acts as an interface between the synchronous clock-driven domain of the back-end electronics of the sub-detectors and the asynchronous data-driven domain of the the commercial networking and processing equipment for the event building and selection.
        The design of the DTH in ATCA standard will be outlined and results from measurements with the
        prototype board will be presented. Firstly, results on distribution of clock, trigger and fast-control data will be shown. Secondly, results on the data flow from aggregating back-end electronics end-points into sub-events and transfer over simplified TCP/IP from the FPGA in to multiple 100 Gbps Ethernet network ports and PC hosts will be covered.

        Speaker: Marc Dobson (CERN)
    • 14:00 15:30
      Track 2 – Offline Computing: Reconstruction and Performance Riverbank R6

      Riverbank R6

      Adelaide Convention Centre

      Convener: Chiara Ilaria Rovelli (Sapienza Universita e INFN, Roma I (IT))
      • 14:00
        Prompt calibration automation at Belle II 15m

        In March 2019 the Belle II detector began collecting data from $e^{+}e^{-}$ collisions at the SuperKEKB electron-positron collider. Belle II aims to collect a data sample 50 times larger than the previous generation of B-Factories. For Belle II analyses to be competitive it is crucial that calibration constants for this data are calculated promptly prior to the main data reconstruction.

        To accomplish this goal a Python plugin package has been developed based on the open-source Apache Airflow package; using Directed Acyclic Graphs (DAGs) to describe the ordering of processes and Flask to provide admin webpages. DAGs for calibration process submission, monitoring of incoming data files, and validation of calibration constants have all been created to help automate the calibration procedure. Connections to the DIRAC grid services are used to synchronize user accounts so that a secure single sign-on for Belle II collaborative services is maintained. A Flask plugin has also been developed to extend and complement the built-in Airflow admin and monitoring webpages. Including adding role-based access using the X.509 certificates and Virtual Organization Membership Service (VOMS) roles of Belle II users.

        We will present the design of this software, and how this new software package is being used in the early stages of Belle II data taking.

        Speaker: Martin Sevior (University of Melbourne (AU))
      • 14:15
        Alignment for the first precision measurements at Belle II 15m

        On March 25th 2019, the Belle II detector recorded the first collisions
        delivered by the SuperKEKB accelerator. This marked the beginning of the
        physics run with vertex detector.

        The vertex detector was aligned initially with cosmic ray tracks without
        magnetic field simultaneously with the drift chamber. The alignment
        method is based on Millepede II and the General Broken Lines track model
        and includes also the muon system alignment and part of drift chamber
        calibrations. To control weak modes, we employ sensitive validation tools and
        various track samples can be used as alignment input, from straight cosmic tracks to mass-constrained decays.

        With increasing luminosity and experience, the alignment is approaching
        the target performance, crucial for the first physics analyses in the
        era of Super-B-Factories. We will present the software framework for the
        detector calibration and alignment, the results from the first
        physics run and the prospects in view of the experience with the first data.

        Speakers: Tadeas Bilka (Charles University, Prague), Tadeas Bilka (Charles University (CZ))
      • 14:30
        Performance of Belle II tracking on collision data 15m

        The tracking system of Belle II consists of a silicon vertex detector (VXD) and a cylindrical drift chamber (CDC), both operating in a magnetic field created by the main solenoid of 1.5 T and final focusing magnets.  The Belle II VXD is a combined tracking system composed by two layers of pixel detectors married with four layers of double sided silicon strip sensors (SVD). The drift chamber consists of 56 layers of sense wires, arranged in interleaved axial and stereo superlayers, to assist track finding and provide full 3D tracking. The tracking algorithms employed at Belle II are based on a standalone reconstruction in SVD and CDC as well as on a combination of the two approaches. The tracking reconstruction is tested on the collision data collected in 2018 and 2019, the results are reported in this talk.

        Speakers: Petar Kevin Rados, Petar Kevin Rados (DESY)
      • 14:45
        Improvements to ATLAS primary vertexing reconstruction for LHC Run 3 15m

        The increasing track multiplicity in ATLAS poses new challenges for primary vertex reconstruction software, where it is expected to reach over 70 inelastic proton-proton collisions per beam crossing during Run-2 of the LHC and even more extreme vertex density in the next upcoming Runs.
        In order to address these challenges, two new tools were adapted.
        The first is the Gaussian track density seed finder, a simple yet powerful analytic model of the track density along the beam axis in order to locate candidate vertices
        The second is the Adaptive Multi Vertex Finder, a global approach to vertex finding and fitting, which deploys the same adaptive vertex fitting technique as the Iterative Vertex Finder algorithm used in Run-2. This allows vertices to compete for nearby tracks in parallel, in order to take into account the vertex structure of the event.
        This talk summarises the optimization and expected performance of the Adaptive Multi Vertex Finder algorithm and software, for conditions foreseen for Run-3 of the LHC. These studies are coupled to a newly optimised vertexing seeder algorithm and further performance studies in the ITk scenario.

        Speaker: Dave Casper (University of California Irvine (US))
      • 15:00
        Experience supporting Belle II CDB server Infrastructure for Phase 3 15m

        The Belle II experiment is a leading world class B-physics experiment. In 2017 BNL became a member of the Belle II collaboration taking responsibility to maintain and develop the Conditions Database (CDB)—an archive of the detector’s conditions at the time of each recorded collision. This database tracks millions of variables—for example, the detector’s level of electronic noise, millimeter-scale movements of the detector due to the strong magnetic field, and variations in electronic response due to small temperature changes—all of which need to be properly taken into account to make sense of Belle II’s measurements. The conditions database was built as an HTTP REST service using tools such as Swagger for the API interface development, Payara for the Java EE application server, and Squid for the caching proxy. This article presents the CDB design, the deployment and continuous development at BNL, including the changes to workflows, authentication and API triggered by the experience of the first data taking. It will as well present details about the capabilities and performance during operation in 2018-2019.

        Speaker: Carlos Fernando Gamboa (Brookhaven National Laboratory (US))
      • 15:15
        The CMS Run Registry: Data Certification and Publication System tool 15m

        The Run Registry of the Compact Muon Solenoid (CMS) experiment at the LHC is the central tool to keep track of the results from the data quality monitoring scrutiny carried out by the collaboration. Recently it has been upgraded for the upcoming Run3 of LHC to a new web application which will replace the current version successfully used during Run1 and Run2. It consists of a Javascript web application frontend which connects to an PostgreSQL database in the backend. The data quality information is recorded for each Lumi Section, which corresponds approximately to 23 seconds of data taking at CMS. The new platform incorporates user feedback on shortcoming and support for new use cases like Machine Learning and interfaces. In this presentation we describe the architecture and the the commissioning of the service over the past months and the rest of long shutdown 2.

        Speaker: Fabio Alberto Espinosa Burbano (Massachusetts Inst. of Technology (US))
    • 14:00 15:30
      Track 3 – Middleware and Distributed Computing: Information Systems, BOINC & User Analysis Riverbank R3

      Riverbank R3

      Adelaide Convention Centre

      Convener: Tomoe Kishimoto (University of Tokyo (JP))
      • 14:00
        Grid Information Systems: Past, Present and Future 15m

        Grid information systems enable the discovery of resources in a Grid computing infrastructure and provide further information about their structure and state.
        The original concepts for a grid information system were defined over 20 years ago and the GLUE 2.0 information model specification was published 10 years ago.
        This contribution describes the current status and highlights the changes over the years.
        It provides an overview of the today's usage from analysing the system logs and compares this with results from over a decade ago.
        A critical analysis of the system is provided with lessons learnt and some perspectives for the future.

        Speaker: Markus Schulz (CERN)
      • 14:15
        Evolution of the WLCG Information Infrastructure 15m

        The WLCG project aimed to develop, build, and maintain a global computing facility for storage and analysis of the LHC data. While currently most of the LHC computing resources are being provided by the classical grid sites, over last years the LHC experiments have been using more and more public clouds and HPCs, and this trend will certainly increase. The heterogeneity of the LHC computing resources is not limited to the procurement mode. It also implies variety of storage solutions and types of computer architecture which represent new challenges for the topology and configuration description of the LHC computing resources. The WLCG Information infrastructure has to evolve in order to meet these challenges and to be flexible enough to follow technology innovation. It should provide a complete and reliable description of all types of the storage and computing resources to ensure their effective use. This implies changes at all levels, starting from the primary information providers, through data publishing, transportation mechanism and central aggregators. The contribution describes proposed changes in the WLCG Information infrastructure, their implementation and deployment.

        Speaker: Julia Andreeva (CERN)
      • 14:30
        CRIC: Computing Resource Information Catalogue as a unified topology system for a large scale, heterogeneous and dynamic computing infrastructure 15m

        CRIC is a high-level information system which provides flexible, reliable and complete topology and configuration description for a large scale distributed heterogeneous computing infrastructure. CRIC aims to facilitate distributed computing operations for the LHC experiments and consolidate WLCG topology information. It aggregates information coming from various low-level information sources and complements topology description with experiment-specific data structures and settings required by the LHC VOs in order to exploit computing resources.

        Being an experiment-oriented but still experiment-independent information middleware, CRIC offers a generic solution, a suitable framework with appropriate interfaces implemented, which can be successfully applied on the global WLCG level, for a particular LHC experiment, for instance, CMS or ATLAS, or even for a special task. For example, a dedicated CRIC instance has been built to support transfer tests performed by DOMA Third Party Copy working group. Moreover, extensibility and flexibility of the system allow CRIC to follow technology evolution and easily implement concepts required to describe new types of computing and storage resources.

        The contribution describes overall CRIC architecture, plug-in based implementation of the CRIC components as well as recent developments and future plans.

        Speaker: Alexey Anisenkov (Budker Institute of Nuclear Physics (RU))
      • 14:45
        Exploiting CRIC to streamline the configuration management of glideinWMS factories for CMS support 15m

        GlideinWMS is a workload management and provisioning system that allows sharing computing resources distributed over independent sites. Based on the requests made by glideinWMS Frontends, a dynamically sized pool of resources is created by glideinWMS pilot Factories via pilot job submission to resource sites' computing elements. More than 400 computing elements (CE) are currently serving more than 10 virtual organizations through glideinWMS, with CMS being the biggest user with 230 CEs. The complex configurations of the parameters definining resource requests, as submitted to those CEs, have been historically managed by manually editing a set of different xml files. New possibilities arise with CMS adopting the Computing Resource Information Catalogue (CRIC), an information system that collects, aggregates, stores, and exposes, among other things, computing resource data coming from various data providers. The talk will describe the challenges faced when CMS started to use CRIC to automatically generate the glideinWMS factory configurations. The architecture of the prototype, and the ancillary tools developed to ease this transition, will be discussed. Finally, future plans and milestones will be outlined.

        Speaker: James Letts (Univ. of California San Diego (US))
      • 15:00
        Distributed data analysis with ROOT RDataFrame 15m

        Widespread distributed processing of big datasets has been around for more than a decade now thanks to Hadoop, but only recently higher-level abstractions have been proposed for programmers to easily operate on those datasets, e.g. Spark. ROOT has joined that trend with its RDataFrame tool for declarative analysis, which currently supports local multi-threaded parallelisation. However, RDataFrame’s programming model is general enough to accommodate multiple implementations or backends: users could write their code once and execute it as is locally or distributedly, just by selecting the corresponding backend.

        This abstract introduces PyRDF, a new python library developed on top of RDataFrame to seamlessly switch from local to distributed environments in a transparent way for users. Programmers are provided with ergonomic interfaces, integrated with web-based services, which allow to dynamically plug in new resources, as well as to write, execute, monitor and debug distributed applications in the most intuitive way possible.

        Speaker: Enric Tejedor Saavedra (CERN)
      • 15:15
        Adapting ATLAS@Home to trusted and semi-trusted resources 15m

        ATLAS@Home is a volunteer computing project which enables members of the public to contribute computing power to run simulations of the ATLAS experiment at CERN. The computing resources provided to ATLAS@Home increasingly come not only from traditional volunteers, but from data centres or office computers at institutes associated to ATLAS. The design of ATLAS@Home was built around not giving out sensitive credentials to volunteers, which means that a sandbox is needed to bridge data transfers between trusted and untrusted domains. As the scale of ATLAS@Home increases, this sandbox becomes a potential data management bottleneck. This paper explores solutions to this problem based on relaxing the constraints of sending credentials to trusted volunteers, allowing direct data transfer to grid storage and avoiding the intermediate sandbox. Fully trusted resources such as grid worker nodes can run with full access to grid storage, whereas semi-trusted resources such as student desktops can be provided with "macaroons": time-limited access tokens which can only be used for specific files. The steps towards implementing these solutions as well as initial results with real ATLAS simulation tasks are discussed along with the experience gained so far and the next steps in the project.

        Speaker: David Cameron (University of Oslo (NO))
    • 14:00 15:30
      Track 4 – Data Organisation, Management and Access: storage systems evolution and challenges Riverbank R8

      Riverbank R8

      Adelaide Convention Centre

      Convener: Alessandra Forti (University of Manchester (GB))
      • 14:00
        Evolution of the S3 service at CERN as a storage backend for infrastructure services and software repositories 15m

        The S3 service at CERN (S3.CERN.CH) is a horizontally scalable object storage system built with a flexible number of virtual RADOS Gateways on top of a conventional Ceph cluster. A Traefik load balancing frontend (operated via Nomad and Consul) redirects HTTP traffic to the RGW backends, and LogStash publishes to ElasticSearch for monitoring the user traffic. User and quota management is delegated to OpenStack; a novel synchronization mechanism was developed to maintain consistency between Keystone and Ceph.

        The RADOS backend went through some notable operational exercises, including massive data rebalancing, a FileStore to BlueStore migration, and the addition of SSDs for bucket indexes; these operations gave us increased capacity and up to 50x speedup for some performance metrics.

        The usage of S3 at CERN has evolved in the recent years. The earliest adopters of S3 were the ATLAS Event Service and CMS BOINC use-cases; the latter takes advantage of S3 pre-signed URLs to offer a secure storage to untrusted remote user PCs. More recently our IT applications demanded S3 object storage: Gitlab and Nexus artifacts, ElasticSearch backups, Kubernetes and others. And our WLCG-specific software is also increasingly taking advantage of the service: CVMFS stratum zero data is a natural fit, and S3 combined with Restic offers a compelling $HOME backup service. Lastly, with the decommissioning of the CERN data centre in Hungary, disaster recovery requirements have motivated a second S3 region on the CERN Prevessin site, with async RGW replication for important buckets.

        Speaker: Enrico Bocchi (CERN)
      • 14:15
        EOS architectural evolution and strategic development directions 15m

        EOS is the main storage system at CERN providing hundreds of PB of capacity to both physics experiments and also regular users of the CERN infrastructure. Since its first deployment in 2010, EOS has evolved and adapted to the challenges posed by ever increasing requirements for storage capacity, user friendly POSIX-like interactive experience and new paradigms like collaborative applications along with sync and share capabilities.

        Overcoming these challenges at various levels of the software stack meant coming up with a new architecture for the namespace subsystem, completely redesigning the EOS FUSE module and adapting the rest of the components like draining, LRU engine, file system consistency check and others, to ensure a stable and predictable performance. In this paper we are going to detail the issues that triggered all these changes along with the software design choices that we made.

        In the last part of the paper, we move our focus to the areas that need immediate improvements in order to ensure a seamless experience for the end-user along with increased over-all availability of the service. Some of these changes have far reaching effects and are aimed at simplifying both the deployment model but more importantly the operational load when dealing with (non/)transient errors in a system managing thousands of disks.

        Speaker: Andreas Joachim Peters (CERN)
      • 14:30
        Erasure Coding for production in the EOS Open Storage system 15m

        The storage group of CERN IT operates more than 20 individual EOS storage services with a raw data storage volume of more than 280 PB. Storage space is a major cost factor in HEP computing and the planned future LHC Run 3 and 4 increase storage space demands by at least an order of magnitude.

        A cost effective storage model providing durability is Erasure Coding (EC). The decommissioning of CERN's remote computer center (Wigner/Budapest) allows to reconsider the currently configured dual-replica strategy where EOS provides one replica in each computer center.

        EOS allows to configure EC on a per file bases and exposes four different redundancy levels with single, dual, triple and fourfold parity to select different quality of service and variable costs.

        This paper will highlight tests which have been performed to migrate files on a production instance from dual-replica to various EC profiles. It will discuss performance and operational impact, and highlight various policy scenarios to select the best file layout with respect to IO patterns, file age and file size.

        We will conclude with the current status and future optimizations, an evaluation of cost savings and discuss an erasure encoded EOS setup as a possible tape storage replacement.

        Speaker: Andreas Joachim Peters (CERN)
      • 14:45
        The ATLAS EventIndex for LHC Run 3 15m

        The ATLAS Event Index was designed in 2012-2013 to provide a global event catalogue and limited event-level metadata for ATLAS analysis groups and users during LHC Run 2 (2015-2018). It provides a good and reliable service for the initial use cases (mainly event picking) and several additional ones, such as production consistency checks, duplicate event detection and measurements of the overlaps of trigger chains and derivation datasets. LHC Run 3, starting in 2021, will see increased data-taking and simulation production rates, with which the current infrastructure would still cope but may be stretched to its limits by the end of Run 3. This talk describes the implementation of a new core storage service that will be able to provide at least the same functionality as the current one for increased data ingestion and search rates and with increasing volumes of stored data. It is based on a set of HBase tables, with schemas derived from the current Oracle implementation, coupled to Apache Phoenix for data access; in this way we will add to the advantages of a BigData based storage system the possibility of SQL as well as NoSQL data access, allowing us to re-use most of the existing code for metadata integration.

        Speaker: Dario Barberis (Università e INFN Genova (IT))
      • 15:00
        Testing the performance of data transfer protocols on long-haul high bandwidth paths. 15m

        This paper describes the work done to test the performance of several current data transfer protocols. The work was carried out as part of the AENEAS Horizon 2020 project in collaboration with the DOMA project and investigated the interactions between the application, the transfer protocol, TCP/IP and the network elements. When operational, the two telescopes in Australia and South Africa that will form the SKA will each need to be able to transfer 1 Petabyte of data per day to various regional centres situated around the world that will extract the science from these data products. These requirements to transfer and access data are very similar to those of the current and future LHC experiments, and this has helped to form the collaboration between the particle physics and radio astronomy communities.
        Measurements were made using high performance server nodes with NVME and RAM disks that were directly connected with 100 Gigabit links to the NRENs and the GÉANT in Europe, and to AARNet in Canberra and the Murchison Royal Observatory in Western Australia.
        Some understanding of the disk to disk transfer rates was gained by measuring the read and write speeds between disk and memory for the file systems in use, and also the observed memory to memory network performance between the hosts used. In addition, some of the open source data transfer and network performance tools were instrumented to provide more insight on the observed rates and performance.
        The measurements were made over various 100 Gigabit links, including dedicated and shared links, with round trip times (RTT) of 7.5, 15, 30 and 300 ms. The disk and file system configurations used included xfs over NVME disks in RAID0, a zfs pool of NVME disks, xfs on RAM disks, and systems at CERN and DESY. The following application and protocols were compared: GridFTP with globus-url-copy, Xrootd with xrdcp using xrd protocol, Xrootd with curl using http protocol, Xrootd with davix using http protocol, and dCache with davix using http protocol.
        The results and analysis of the data from the sub-systems’ and application performance are presented and discussed in relation to the end-to-end disk transfer rates observed. The paper concluded with some recommendations for improving performance and welcomes open discussion. The authors acknowledge the help from members of the DOMA Project.

        Speaker: Dr Richard Hughes-Jones (GEANT Association)
      • 15:15
        Using Graph Databases in HEP 15m

        Data in HEP are usually stored in tuples (tables), trees, nested tuples (trees of tuples) or relational (SQL-like) databases, with or without a defined schema. But many of our data have a graph structure without a schema, or with a weakly imposed schema. They consist of entities with relations, some of which are known in advance, but many are created later, as needs evolve. Such structures are not well covered by relational (SQL) databases. We don't only need the possibility to add new data with predefined relations. We also need to add new relations. Graph databases have existed for a long time but they have only recently matured, mostly thanks to Big Data and Machine Learning. There are now very good implementations and de-facto standards widely available.
        The difference between SQL and Graph Database is similar to the difference between Fortran and C++. On one side, a rigid system, which can be very optimized. On the other side, a flexible dynamical system, which allows expressing of complex structures. Graph Database is a new synthesis of object-oriented and relational databases. It allows the expression of a web of objects without the volatility of the object-oriented world. It captures only essential relations, it dosen't keep a complete object dump. Migration to Graph Database means moving structure from data to code, together with migration from imperative to declarative semantics (things don't 'happen', they 'exist'). Data, stored in a structured Graph Database also allows new ways of easy analysis with the help of the Non-Euclidian Machine Learning methods. Those methods are not based on the geometrical structure of the problem domain, but on its topology. This makes the Graph Database structure particularly useful for such an approach.
        This presentation will describe the basic principles of the Graph Database together with an overview of existing standards and implementations. The usefulness and usability will be demonstrated for the concrete example of the ATLAS EventIndex in two approaches - as the full storage (all data are in the Graph Database) and meta storage (a layer of schema-less graph-like data implemented on top of either NoSQL or relational storage). The usability, the interfaces with the surrounding framework and the performance of those solution will be discussed. The possible more general usefulness for generic experiments' storage will be also discussed. Some examples of using Graph-like data for simple Machine-Learning processing will be also shown.

        Speaker: Julius Hrivnac (Centre National de la Recherche Scientifique (FR))
    • 14:00 15:30
      Track 5 – Software Development: Common tools 2: Messaging, declarative frameworks, optimizers Riverbank R2

      Riverbank R2

      Adelaide Convention Centre

      Convener: Jason Webb (Brookhaven National Lab)
      • 14:00
        Event-driven RDMA network communication in the ATLAS DAQ system with NetIO 15m

        NetIO is a network communication library that enables distributed applications to exchange messages using high-level communication patterns such as publish/subscribe. NetIO is based on libfabric and supports various types of RDMA networks, for example, Infiniband, RoCE, or OmniPath. NetIO is currently being used in the data acquisition chain of the ATLAS experiment.

        Major parts of NetIO were recently rewritten using a novel, event-driven approach. All actions are processed asynchronously by a single-threaded central event loop. The event loop is backed by the Linux epoll system. The event-driven design implies that software written with NetIO uses callbacks to react to events.

        The motivation for the architectural modifications to NetIO was to improve processing efficiency. Initial benchmarks show that the updated NetIO implementation yields the same or higher throughput, while the CPU resource utilization is reduced by an order of magnitude. The cause for this efficiency gain is largely due to significantly reduced thread synchronization, that became obsolete in the event-driven approach.

        The paper will show this architecture is very suitable for IO-heavy workloads that are typically found in DAQ systems of High-Energy Physics experiments. The event-driven architecture will be explained in detail and compared with the original NetIO. The challenges of writing event-driven code are identified. A performance study of the event-driven NetIO in comparison with the original implementation as well as other RDMA networking solutions like MPI will be given.

        Speaker: Jorn Schumacher (CERN)
      • 14:15
        dCache -- Efficient Message Encoding For Inter-Service Communication 15m

        As a well established, large-scale distributed storage system, dCache is required to manage and serve huge amount of data for WLHC experiments and beyond. Based on a microservices-like architecture, dCache is built as a modular system distributed, where each component provides a different core functionality. These services communicate by passing serialized messages of dynamic types to each other, a core behavior whose performance properties can consequently affect the entire system.

        The usage of Java Object Serialization for encoding messages has increasingly presented as no longer being sufficiently performant. A metric for evaluating message object graph complexity is introduced and the performance of several serialization techniques is evaluated accordingly. The mapping of a dynamic object tree to a fixed format, that can be serialized efficiently, is investigated.

        In this presentation we introduce the flexible integration of a new message serialization format into dCache, which enables the simultaneous support of different encoding techniques, allowing for a gradual transition.

        Speaker: Mr Tigran Mkrtchyan (DESY)
      • 14:30
        EOS Erasure Coding plug-in as a case study for the XRootD client declarative API 15m

        One of the key components of the XRootD software framework is the C++ implementation of the XRootD client. As the foundation of client binaries, XRootD Posix API and the Python API, it is widely used in LHC experiments’ frameworks as well as on server side in the XCache and EOS. In order to facilitate new developments the XRootD client API has been extended to be in line with modern C++ programming practices.
        In this contribution we report on the new XRootD client declarative API inspired by C++ ranges v3. The new utility offers superior composability of asynchronous operations through operation pipelining, and support for standard callbacks (lambdas, std::futures). As a case study we consider the EOS erasure coding plug-in.

        Speaker: Andrew Bohdan Hanushevsky (SLAC National Accelerator Laboratory (US))
      • 14:45
        Analysis Tools for the VyPR Performance Analysis Framework for Python 15m

        VyPR (http://cern.ch/vypr) is a framework being developed with the aim of automating as much as possible the performance analysis of Python programs. To achieve this, it uses an analysis-by-specification approach; developers specify the performance requirements of their programs (without any modifications of the source code) and such requirements are checked at runtime. VyPR then provides tools which allow developers to perform detailed analyses of the performance of their code. Such analyses can include determining the common paths taken to reach badly performing parts of code, deciding whether a single path through code led to variations in time taken by future observations, and more.

        This paper describes the developments that have taken place in the past year on VyPR’s analysis tools to yield a Python shell-based analysis library, and a web-based application. It concludes by demonstrating the use of the analysis tools on the CMS Experiment’s Conditions Upload service.

        Speaker: Joshua Heneage Dawes (University of Manchester (GB))
      • 15:00
        Vectorized, imperative, and declarative processing of Awkward Arrays 15m

        Over the past two years, the uproot library has become widely adopted among particle physicists doing analysis in Python. Rather than presenting an event model, uproot gives the user an array for each particle attribute. In case of multiple particles per event, this array is jagged: an array of unequal-length subarrays. Data structures and operations for manipulating jagged arrays are provided by the awkward-array library, which also includes array types for nested structures, nullability, and heterogeneity.

        The primary mode of awkward-array manipulation is vectorized in the sense of a Single Python Instruction on Multiple Data (“virtual machine SIMD”). Many implementations of these high-level instructions are also vectorized in the hardware sense. But whereas the implicit loops of vectorized instructions make some calculations easier, they make others harder, especially algorithms that iterate until a convergence condition is met.

        To support algorithms with explicit for loops, we have extended awkward-array to work with Numba, a popular Python JIT-compiler. Vectorized and imperative programming styles may now be used interchangeably on the same awkward-array structures, with no loss of performance.

        These two programming styles, column-first vectorized and row-first imperative, differ in their order of execution. We will also show progress on a declarative interface, which doesn’t specify the execution order, allowing the implementation to optimize it independently of the content of the physics analysis itself.

        Speaker: Jim Pivarski (Princeton University)
      • 15:15
        Impact of different compilers and build types on Geant4 simulation execution time 15m

        Experimental observations and advanced computer simulations in High Energy Physics (HEP) paved way for the recent discoveries at the Large Hadron Collider (LHC) at CERN. Currently, Monte Carlo simulations account for a very significant amount of computational resources of the Worldwide LHC Computing Grid (WLCG).
        In looking at the recent trends in modern computer architectures we see a significant deficit in expected growth in performance. Coupled with the increasing compute demand for High Luminosity (HL-LHC) run, it becomes vital to address this shortfall with more efficient simulation.

        The simulation software for particle tracking algorithms of the LHC experiments predominantly relies on the Geant4 simulation toolkit. The Geant4 framework can be built using either dynamic or static libraries, the former being the more widely used approach. This study focuses on evaluating the impact of having libraries statically vs dynamically linked, on the simulation software’s execution time.

        Two versions of the GCC compiler, namely 4.8.5 and 8.2.0, have been used for these investigations. In addition, a comparison between four optimization levels (Os, O1, O2 and O3) have also been performed. The results show that the static approach, for both the GCC versions considered, reduces the execution time by more than 10% in some cases. Regardless of the build approach, switching from GCC 4.8.5 to GCC 8.2.0 results in an average of 30% improvement in the execution time. In particular, a static build with GCC 8.2.0 leads to an improvement of almost 34% with respect to the default configuration (GCC 4.8.5, dynamic, O2). The different GCC optimizations do not seem to have visible effects on the execution time.

        Speaker: Caterina Marcon (Lund University (SE))
    • 14:00 15:30
      Track 6 – Physics Analysis: Event Reconstruction and Selection methods Hall G

      Hall G

      Adelaide Convention Centre

      Convener: Maurizio Pierini (CERN)
      • 14:00
        Lepton identification in Belle II using observables from the elctromagnetic calorimeter and precision trackers. 15m

        We present a major overhaul to lepton identification for the Belle II experiment, based on a novel multi-variate classification algorithm.

        A key topic in the Belle II physics programme is the study of semi-tauonic B decays to test lepton flavour universality violation, such as $B\rightarrow D^{*}\tau\nu$.  The analysis of this decay relies on the capability of correctly separating low momentum, lepton candidates ($e$, $\mu$) in the decay of the $\tau$ from hadronic background. At the mean momentum of $\leq 500$ MeV/c, muons do not reach the dedicated muon detector (KLM) , therefore methods must be developed to identify them with the electromagnetic calorimeter (ECL). Furthermore, at low momenta electrons suffer significant energy losses due to bremsstrahlung, making them more easily mimicked by hadrons.

        Boosted decision trees are trained combining measurements from the ECL and tracking system. The chosen observables are sensitive to the different physics that governs interactions of hadrons, electrons and muons with the calorimeter crystals. Dedicated classifiers are used in various detector regions and lepton momentum ranges. The tree output is eventually combined with other classifiers that rely upon independent measurements from other sub-detectors.

        Using simulation, the performance of the new algorithm is compared against the method used for analysis of the 2018 Belle II data, namely a likelihood discriminator based on the ratio of energy measured in the ECL over the momentum measured by the trackers. In the critical low momentum region, we largely improve the lepton-pion separation power, increasing background rejection up to 50% for the same electron/muon signal efficiency.

        The algorithm has been integrated into the Belle II Analysis Software Framework, and is used in the processing of the 2019 collision dataset. Its performance in the 2019 run will be covered in the presentation.

        Speaker: Dr Marco Milesi (The University of Melbourne, Belle II Experiment)
      • 14:15
        Applications of the Recursive Jigsaw Reconstruction Method 15m

        The Recursive Jigsaw Reconstruction method is a technique to analyze reconstructed particles in the presence of kinematic and combinatoric unknowns which are associated with unmeasured or indistinguishable particles. By factorizing the unknowns according to an assumed topology and applying fixed algorithmic choices - Jigsaw Rules, we are able to approximately reconstruct rest frames throughout the assumed topology. In this talk I will provide an overview of the applications of this technique.

        Speaker: Jason Oliver (University of Adelaide (AU))
      • 14:45
        Optimising HEP parameter fits through MC weight derivative regression 15m

        HEP event selection is traditionally considered a binary classification problem, involving the dichotomous categories of signal and background. In distribution fits for particle masses or couplings, however, signal events are not all equivalent, as the signal differential cross section has different sensitivities to the measured parameter in different regions of phase space. In this talk, I describe a mathematical framework for the evaluation and optimization of HEP parameter fits, where this sensitivity is defined on an event-by-event basis, and for MC events it is modeled in terms of their MC weight derivatives with respect to the measured parameter. Minimising the statistical error on a measurement implies the need to resolve (i.e. separate) events with different sensitivities, which ultimately represents a non-dichotomous classification problem. Since MC weight derivatives are not available for real data, the practical strategy I suggest consists in training a regressor of weight derivatives against MC events, and then using it as an optimal partitioning variable for 1-dimensional fits of data events. I discuss some features and limitations of this approach and I show examples from a simple toy model. This research is an extension of the work I presented at CHEP2018: in particular, event-by-event sensitivities allow the exact computation of the "FIP" ratio (a metric in [0,1]) between the Fisher information obtained from an analysis and the maximum information that could possibly be obtained with an ideal detector. Using this expression, I discuss the relationship between FIP and two metrics commonly used in meteorology (Brier score and MSE), and the importance of "sharpness" in both domains. I finally point out that HEP distribution fits should be optimized and evaluated using probabilistic metrics (like FIP or MSE), whereas ranking metrics (like AUC) or threshold metrics (like "accuracy") are of limited relevance for these specific problems.

        Speaker: Andrea Valassi (CERN)
      • 15:00
        Missing mass method for reconstruction of short-lived particles 15m

        The main purpose of modern experiments with heavy ions is a comprehensive study of the QCD phase diagram in the field of quark-gluon plasma (QGP) and the possible phase transition to the QGP phase.

        One of the possible signals of QGP formation is an increase in the production of strange particles. Reconstruction of $\Sigma$ particles together with other strange particles completes the picture of strangeness production. $\Sigma^+$ and $\Sigma^-$ have all decay modes with at least one neutral daughter, which cannot be registered in a detector system.

        To identify them, the missing mass method is proposed: a) to reconstruct the tracks of mother particles ($\Sigma^-$) and charged daughter particles ($\pi^-$) in the tracking system; b) to reconstruct the neutral daughter particle ($n$) from these trajectories; c) to build the mother particle from the charged and reconstructed neutral daughter particles and determine its mass spectrum.

        The missing mass method is also used to reconstruct other strange particles. A total of 18 such decays with a neutral daughter particle are included in the physics analysis. The details of the method implementation are given and the results of its application to the simulated data of the CBM (FAIR/GSI, Germany) experiment are discussed.

        The method and its implementation may be of interest for other experiments in both heavy ion and particle physics.

        Speaker: Prof. Ivan Kisel (Johann-Wolfgang-Goethe Univ. (DE))
      • 15:15
        Aligning the MATHUSLA Detector Test Stand with Tensor Flow 15m

        MATHUSLA has been proposed as a detector that sits over 100 m from an LHC interaction point, on the surface, to look for ultra long-lived particles. A test stand was constructed with two layers of scintillator paddles and six layers of RPCs, on loan from the DZERO and the Argo-YBJ experiments. Downward and upward going tracks from cosmic ray data and muons from the interaction point have been reconstructed. To align the detector we have used a large sample of tracks and done a simultaneous fit to find the location of the detectors and the track parameters themselves. We used TensorFlow to drive the fit and align the detector. Previous work directly translated C++ code into TensorFlow primitives. In this updated work we have moved towards more fully using TensorFlow primitives as well as the GPU capabilities of TensorFlow. All aspects of this work will be discussed, including the effort to move away from the C++ structured code to the TensorFlow tensors and more powerful primitives.

        Speaker: Gordon Watts (University of Washington (US))
    • 14:00 15:30
      Track 7 – Facilities, Clouds and Containers: Network technologies Riverbank R4

      Riverbank R4

      Adelaide Convention Centre

      Conveners: Oksana Shadura (University of Nebraska Lincoln (US)), Sang Un Ahn (Korea Institute of Science & Technology Information (KR))
      • 14:00
        Readying CERN for connected device era 15m

        Whether you consider “IoT” as a real thing or a buzzword, there’s no doubt that connected devices, data analysis and automation are transforming industry. CERN is no exception: a network of LoRa-based radiation monitors has recently been deployed and there is a growing interest in the advantages connected devices could bring—to accelerator operations just as much as to building management.

        Connected devices bring risks as well as advantages, however, and the last thing any business needs is an unsafe, uncoordinated and unmanaged sensor environment. To support the deployment of the LoRa-based radiation sensors, CERN has established both a Low Power Wireless Area Network (LPWAN) to complement the existing wired and wireless networks and a service infrastructure to manage the provisioning, orchestration, data transfer, security and operation for connected devices.

        This presentation will describe CERN’s LPWAN infrastructure and our connected-device support architecture and set out how we foresee this being used to support devices that connect to higher-bandwidth networks such as Wi-Fi, BLE or 5G.

        Speaker: Rodrigo Sierra (CERN)
      • 14:15
        Unroutable LHCONE traffic 15m

        This talk explores the methods and results confirming the baseline assumption that LHCONE traffic is science traffic. The LHCONE (LHC Open Network Environment) is a network conceived to support globally distributed collaborative science. The LHCONE connects thousands of researchers to LHC data sets at hundreds of universities and labs performing analysis within the global collaboration. It is “Open” to all levels of the LHC as well as a short list of approved non-LHC science collaborations. It is distinct from the smaller, tightly integrated and private LHCOPN (Optical Private Network) network which is strictly for “Tier 1” compute centers and used in support of the engineered workflow for LHC data processing, distribution and longtime storage of the baseline datasets. LHCONE satisfies the need for a high performance global data transfer network of networks supporting scientific analysis at universities and science labs.

        Science traffic separation is the hard part
        The separation of science flows from non-science flows, is an essential first step in traffic engineering high performance science networks. Before resources or preference can be applied to more effectively move science data, it is essential to identify and separate the science from non-science traffic. This talk explores the methods and results in detecting traffic in the LHCONE network that does not comply with the Appropriate Use Policy established by the global LHC collaboration.

        LHCONE hosts are high performance
        Through integration of the Science DMZ network model and collaborative software platforms. The data transfer nodes connected to LHCONE are high performing data movers placed on the network edge/Science DMZ and secured precisely according to the applications they support and the purpose they serve.

        LHCONE is at risk of unauthorized use
        Unauthorized use of LHCONE places both the network and the sites using it at risk. The risk takes two forms: Science flows are mixing with non-science flows or unauthorized traffic is being dropped inside LHCONE.

        Identifying unauthorized traffic
        An EDUgain authenticated portal visualizing unauthorized usage will be demonstrated. For keeping track of the frequent changes of LHCONE the underlying database will be maintained and administered cooperatively by the LHC and NREN community.

        Speaker: Mr Bruno Hoeft (Karlsruhe Institute of Technology (KIT))
      • 14:30
        IPv6-only networking on WLCG 15m

        The use of IPv6 on the general internet continues to grow. Several Broadband/Mobile-phone companies, such as T-Mobile in the USA and BT/EE in the UK, now use IPv6-only networking with connectivity to the IPv4 legacy world enabled by the use of NAT64/DNS64/464XLAT. Large companies, such as Facebook, use IPv6-only networking within their internal networks, there being good management and performance reasons for this. The transition of WLCG central and storage services to dual-stack IPv4/IPv6 is progressing well, thus enabling the use of IPv6-only CPU resources as agreed by the WLCG Management Board and presented by us at earlier CHEP conferences.

        During the last year, the HEPiX IPv6 working group has not only been chasing and supporting the transition to dual-stack services, but has also been encouraging network monitoring providers to allow for filtering of plots by the IP protocol used. We have investigated and fixed the reasons for the use of IPv4 between two dual-stack endpoints when IPv6 should be preferred. We present this work and the tests that have been made of IPv6-only CPU showing the successful use of IPv6 protocols in accessing WLCG services.

        The dual-stack deployment does however result in a networking environment which is much more complex than when using just IPv6. Some services, e.g. the EOS storage system at CERN, are using IPv6-only for internal communication, where possible. The group is investigating the removal of the IPv4 protocol in more places. We will present the areas where this could be useful and possible and suggest a timetable for being able to turn off IPv4 in this way.

        Speaker: David Kelsey (Science and Technology Facilities Council STFC (GB))
      • 14:45
        LHCb Online migration to ISC Kea - the modern DHCP server. 15m

        DHCP is an often overlooked, but incredibly important component of the operation of every data center. With constantly scaling and dynamic environments, managing DHCP servers that rely on configuration files, which must be in sync, becomes both expensive engineering wise and slow. The LHCb Online infrastructure currently consists of over 2500 DHCP enabled devices - physical and virtual machines, switches, containers and miscellaneous electronics, with plans for the upcoming upgrade to move to over 6000 DHCP client devices. At the same time, up until recently the mentioned infrastructure was served by ISC DHCP servers, with the only high availablity assurance being the redundant hardware configuration. With view of the modern data center operating practices, aging features of ISC DHCP, readily available solutions for highly available database backends, great benefits to using stateless services and need for easy and effortless scalability and recovery, the LHCb Online team decided to move away to a modern DHCP Server - Kea. For the upcoming upgrade, the LHCb Online team migrated all of our old ISC DHCP infrastructure to ISC Kea. This submission will take a look at the benefits provided by using a modern DHCP server, the pitfalls and problems we experienced during the migration and the benefits acquired by the team from this migration.

        Speaker: Hristo Umaru Mohamed (CERN)
      • 15:00
        Network Capabilities for the HL-LHC Era 15m

        High Energy Physics (HEP) experiments have greatly benefited from a strong relationship with Research and Education (REN) network providers and thanks to the projects such as LHCOPN/LHCONE and REN contributions, have enjoyed significant capacities and high performance networks for some time. Network providers have been able to continually expand their capacities to over-provision the networks relative to the experiments needs and were thus able to cope with the recent rapid growth of the traffic between sites, both in terms of achievable peak transfer rates as well as in total amount of data transferred.

        There are reasons to believe that the network situation will change due to both technological and non-technological reasons starting already in the next few years. Other data-intensive sciences will join with data scales similar to LHC, which will impact not only R&E providers, but also the way end-users are currently utilising the network. In the new multi-science high throughput environment, network provisioning, design and operations will need to evolve to better share and organise the available resources. Apart from the network capacity, other network capabilities will become important as they will offer a way to better organise and more effectively use the available network resources.

        As the scale and complexity of the current networks grow rapidly, new technologies and platforms are being introduced that greatly extend the capabilities of today’s networks. With many of these technologies becoming available, it’s important to understand how we can design, test and develop systems that could enter existing production workflows while at the same time changing something as fundamental as the network that all sites and experiments rely upon. In this talk we’ll provide a high level summary of the white paper produced by the HEPiX NFV Working Group, providing an overview of the current software defined networking landscape, including areas such as cloud native networking, existing and planned projects in the area of programmable networks, smartNICs and related technologies as well as an overview of the mid-term plans of the core R&E network providers.

        Speaker: Shawn Mc Kee (University of Michigan (US))
      • 15:15
        Network in Belle II 15m

        Belle II has started the Phase 3 data taking with a fully quipped detector. The data flow at the maximum luminosity is expected to be 12PB of data/year and will be analysed by a cutting-edge computing infrastructure spread over 26 Countries. Some of the major Computing Centres for HEP in Europe, USA and Canada will store and tackle the second copy of RAW data.
        In this scenario, the international Network Infrastructure for Research plays a key role in supporting and orchestrating all the activities of data analysis and replication. The large-scale data challenge will also take advantage form LHCONE VRF service and the support of Network experts of KEKCC, Belle II sites and NREN. The program of major upgrade in 2019 massively empowered the connection among Japan, Europe and USA over a 100Gb geographic ring.
        In this work, we summarize the network requirements needed to accomplish all the tasks provided by the computing model. We also highlight the status of the major network links that support and advance Belle II. Lastly, we present the results of the last Network Data Challenge campaign performed between KEK and the main RAW Data centres with the additional usage of the Data Transfer Node service provided by GEANT.

        Speaker: Silvio Pardi (INFN)
    • 14:00 15:30
      Track 7 – Facilities, Clouds and Containers: Opportunistic resources Riverbank R7

      Riverbank R7

      Adelaide Convention Centre

      Convener: Christoph Wissing (Deutsches Elektronen-Synchrotron (DE))
      • 14:00
        Lightweight dynamic integration of opportunistic resources 15m

        Dynamic resource provisioning in the WLCG is commonly based on meta-scheduling and the pilot model. For a given set of workflows, a meta-scheduler computes the ideal set of resources; so-called pilot jobs integrate these resources into an overlay batch system, which then processes the initial workflows. While offering a high level of control and precision, the strong coupling between components limits scalability, flexibility and robustness. These shortcommings are more severe when workflows and resources are under limited control - such as a WLCG site executing anonymous pilot jobs on external, non-WLCG resources.

        In order to integrate dynamic resources, the GridKa Tier 1 centre has developed a new approach for dynamic provisioning that is suitable for the WLCG and beyond. By design, our approach decouples the distinct responsibilities of workflow scheduling, resource provisioning and meta-scheduling. Instead of seeking an optimal solution for a coupled scheduling and meta-scheduling problem, we divide the task into composable but isolated, self-balancing domains. Not only does this naturally provide scalability, flexibility and robustness, it also allows us to manage a variety of resources and situations in a common way. We have successfully used our work for provisiong HPC and Cloud resources to the WLCG, as well as managing abstract resources in the form of Multi-Core and Single-Core allocations.

        This contribution discusses the benefits and limitations of our new approach to dynamic resource provisioning and compared to competing approaches.

        Speaker: Max Fischer (Karlsruhe Institute of Technology)
      • 14:15
        External Resources: Clouds and HPCs for the expansion of the ATLAS production system at the Tokyo regional analysis center 15m

        The Tokyo regional analysis center at the International Center for Elementary Particle Physics, the University of Tokyo, is one of the Tier 2 sites for the ATLAS experiment in the Worldwide LHC Computing Grid (WLCG). The current system provides 7,680 CPU cores and 10.56 PB disk storage for WLCG. CERN plans the high-luminosity LHC starting from 2026, which increases the peak luminosity to 5 times compared to the present value in LHC. For the high-luminosity LHC, a requirement of computing resources for each site will be increased. To expand the ATLAS production system at the Tokyo regional analysis center, R&D using external resources has been launched. One kind of external resources is a commercial cloud resource, such as Google Cloud Platform and Amazon AWS. Another resource is the High-Performance Computer (HPC) at the University of Tokyo. In this presentation, the current status of the R&D, the systems for these resources and comparisons of the cost will be reported.

        Speaker: Michiru Kaneda (ICEPP, the University of Tokyo)
      • 14:30
        Effective Dynamic Integration and Utilization of Heterogenous Compute Resources 15m

        Increased operational effectiveness and the dynamic integration of only temporarily available compute resources (opportunistic resources) becomes more and more important in the next decade, due to the scarcity of resources for future high energy physics experiments as well as the desired integration of cloud and high performance computing resources. This results in a more heterogenous compute environment, which gives rise to huge challenges for the computing operation teams of the experiments.

        At the Karlsruhe Institute of Technology we design solutions to tackle these challenges. In order to ensure an efficient utilization of opportunistic resources and unified access to the entire infrastructure, we developed the Transparent Adaptive Resource Dynamic Integration System (TARDIS). A scalable multi-agent resource manager providing interfaces to provision as well as dynamically and transparently integrate resources of various providers into one common overlay batch system. Operational effectiveness is guaranteed by relying on COBalD - the Opportunistic Balancing Daemon and its simple approach of taking into account the utilization and allocation of the different resource types, in order to run the individual workflows on the best-suited resource respectively.

        In this contribution we will present the current status of integrating various HPC centers and cloud providers into the compute infrastructure at the Karlsruhe Institute of Technology as well as our experiences gained in a production environment.

        Speaker: Manuel Giffels (KIT - Karlsruhe Institute of Technology (DE))
      • 14:45
        Offsite Data Processing for the GlueX Experiment 15m

        The Jefferson Lab 12GeV accelerator upgrade completed in 2015 is now producing data at volumes unprecedented for the lab. The resources required to process this data now exceed the capacity of the onsite farm necessitating the use of offsite computing resources for the first time in the history of JLab. GlueX is now utilizing NERSC for raw data production using the new SWIF2 workflow tool developed at JLab. In addition, simulation is being done on the OSG utilizing the same container and distributed file system technologies deployed for NERSC. Details of the workflow will be presented.

        Speaker: David Lawrence (Jefferson Lab)
      • 15:00
        Running HTC and HPC applications opportunistically across private, academic and public clouds 15m

        Access to both High Throughput Computing (HTC) and High Performance Computing (HPC) facilities is vitally important to the fusion community, not only for plasma modelling but also for advanced engineering and design, materials research, rendering, uncertainty quantification and advanced data analytics for engineering operations. The computing requirements are expected to increase as the community prepares for ITER, the next generation facility. Moving to a decentralised computing model is vital for future ITER analysis where no single site will have sufficient resource to run all necessary workflows.

        The Fusion Science Demonstrator in the European Open Science Cloud for Research Pilot Project (EOSCpilot) aimed to demonstrate that the fusion community can make use of distributed cloud resources. PROMINENCE is a platform initially developed within this Science Demonstrator and enables users to transparently exploit idle cloud resources for running scientific workloads. In addition to standard HTC jobs, HPC jobs such as multi-node MPI are supported. All jobs are run in containers to ensure they will reliably run anywhere and are reproduceable. Cloud infrastructure is invisible to users, as all provisioning, includingextensive failure handling, is completely automated. On-premises cloud resources can be utilised and at times of peak demand burst onto external clouds. In addition to the traditional "cloud-bursting" onto a single cloud, PROMINENCE allows for bursting across many clouds in a hierarchical manner, for example bursting from a local private cloud to national research clouds, then across many clouds in the EGI FedCloud federation, and finally to public clouds. Job requirements are also taken into account, so jobs with special requirements, e.g. high memory or access to GPUs, are sent only to appropriate clouds. Several different storage options are available to either allow data to be staged-in and out of jobs using Swift/S3 or to provide POSIX-like access to data, irrespective of where the job is running.

        In this presentation we will describe PROMINENCE, its architecture and the challenges of using many clouds opportunistically. We will also report on our experiences with several fusion use cases.

        Speaker: Dr Andrew Lahiff
      • 15:15
        ATLAS Sim@P1 upgrades during long shutdown two 15m

        The Simulation at Point1 (Sim@P1) project was built in 2013 to take advantage of the ATLAS Trigger and Data Acquisition High Level Trigger (HLT) farm. The HLT farm provides around 100,000 cores, which are critical to ATLAS during data taking. When ATLAS is not recording data, this large compute resource is used to generate and process simulation data for the experiment. At the beginning of the current long shutdown (LS2), the HLT farm including the Sim@P1 infrastructure was upgraded. Previous papers emphasized the need for “simple, reliable, and efficient tools” and assessed various options to quickly switch between data acquisition operation and offline processing. In this contribution we describe the new mechanisms put in place for the opportunistic exploitation of the HLT farm for offline processing and give results from the first months of operation.

        Speaker: Frank Berghaus (University of Victoria (CA))
    • 14:00 15:30
      Track 9 – Exascale Science: Software environment, quantum algorithms, others Riverbank R1

      Riverbank R1

      Adelaide Convention Centre

      Convener: Steven Farrell (Lawrence Berkeley National Lab (US))
      • 14:00
        The SKA Science Data Processor (SDP): final design and getting ready for the construction phase 15m

        The Square Kilometre Array (SKA) project is an international effort to build the world’s largest radio telescope, led by the SKA Organisation based at the Jodrell Bank Observatory near Manchester, UK. The SKA will conduct transformational science to improve our understanding of the Universe and the laws of fundamental physics, monitoring the sky in unprecedented detail and mapping it hundreds of times faster than any current facility. On 12 March 2019, members of 7 countries signed the international treaty that marks the beginning of the establishment of the SKA Observatory as the intergovernmental organisation responsible of the delivery and operations of this new radio telescope. Currently almost all the elements of the SKA have completed its design work and the project is now moving towards the construction phase.

        The Science Data Processor (SDP) element finished its engineering design work at the end of March 2019 completing almost 5 years of designing the computing software and hardware systems responsible of processing the vast amount of data coming from the telescopes (~ 5 Tb/s). It is estimated that the SDP will produce and distribute approx. 1 PB of science-ready products per day to astronomers across the globe. This sheer amount of processing capability will be made possible by dedicated High Performance Computing (HPC) platforms deployed in Australia and South Africa for a combined peak processing power of approx. 250 PFLOPS. This design effort was completed by the SDP consortium, a collaboration of astrophysicists, engineers and computer scientists working in 11 countries worldwide and led by the University of Cambridge.

        This paper will present a status of the SKA project, a summary of the SDP design work and what lies ahead towards the construction phase.

        Speaker: Dr Stephen Ord (CSIRO Astronomy and Space Science)
      • 14:15
        Rapid Development of Accelerated Imaging Algorithms for ASKAP 15m

        The software pipeline for ASKAP has been developed to run on the Galaxy supercomputer as a succession of MPI enabled coarsely parallelised applications. We have been using OpenACC to develop more finely grained parallel applications within the current code base that can utilise GPU accelerators if they are present. Thereby eliminating the overhead of maintaining two versions of the software and without rewriting large sections of the code base. This talk will outline how we determined which elements to accelerate, how we applied openACC to simply develop parallel applications and the benefits, and disadvantages, of this type of development cycle.

        Speaker: Dr Stephen Ord (CSIRO Astronomy and Space Science)
      • 14:30
        Physics beyond the Standard Model in the evolving computing architecture of the KISTI-5 supercomputer 15m

        In November 2018, KISTI-5 supercomputer has launched. It is the heterogeneous machine of 25.3 PF Cray 3112-AA000T with Intel Xeon Phi KNL (Knight Landing) 7250 processor which has 68 cores per processor. The goal of this presentation is to discuss the application and usages of Intel KNL-based system of KISTI-5 supercomputer for physics beyond the Standard Model.

        The world is made of dark energy, dark matter and the Standard Model particles. The Standard Model is the last frontier of universe. Evolving universe is towards a unified description of the nucleus. Let us show some of potential works – physics beyond the Standard Model, simulation, evolving universe and so on.

        First, the Standard Model in particle physics is refined. However, new physics beyond the Standard Model, such as dark matter, requires thousand to million times of simulation events compared to those of the Standard Model. Thus, the development of software is required, especially for the development of simulation tool kits.

        Second, computing is evolving to exascale, which requires the development of the simulation tool kit to accommodate the evolving computing architecture. Therefore, an efficient simulation tool kit is needed. A profiling system is also required to confirm it. In Geant4, a typical simulation tool kit, a profiling system in higher-energy physics areas such as the Large Hardon Collider (LHC) experiment is well developed, contributing to the development of the software. However, profiling systems in the low-energy physics domain are in the beginning stage. For this reason, we have developed and performed it. These profiling systems could be used to confirm the development of software for evolving computing architecture.

        Third, the evolving universe requires thousand to million times of simulation. For this, KISTI-5 supercomputer is used in research for production of exotic nuclei and heavy elements, Abi initio nuclear structure and reactions, and lattice effective theory. Let us show the current status and future plan for these.

        Speaker: Prof. Kihyeon Cho (KISTI)
      • 14:45
        Enabling change for validated exascale fusion nuclear science 15m

        Within the fusion radiation transport community for many years the de facto standard codebase for simulation was and still is MCNP. MCNP suffers from very few community perceived drawbacks having widely validated and verified physics, large user base, simple interface, but the main issue in the age of democratised computing access is prohibitive licence conditions. Thus, if we need to be able to deploy our calculations anywhere without restriction, we must use another transport code. Herein there is reporting of a new tool, csg2csg, which allows MCNP geometry to be translated to other formats including FLUKA, Serpent2, PHITS and OpenMC. This tool allows geometry and materials to be translated between forms, which facilitates the performance of standardised tests between codes, e.g. shielding benchmarks in SINBAD and criticality benchmarks in ICSBEP. Examples of fusion & HEP relevant validation and comparisons will be shown.

        Speaker: Prof. Andrew Davis (United Kingdom Atomic Energy Authority)
      • 15:00
        Testing GPU inverters on ROCm 15m

        The open source ROCm platform for GPU computing provides an uniform framework to support both the NVIDIA and AMD GPUs, and also the possibility to porting the CUDA code to the ROCm-compatible one. We will present the porting progress on the Overlap fermion inverter (GWU-code) based on thrust and also a general inverter package - QUDA.

        Speaker: Yujiang Bi (Institute of High Energy Physics, Chinese Academy of Sciences)
    • 15:30 16:30
      Posters: B Hall F

      Hall F

      Adelaide Convention Centre

      • 15:30
        BESIII utilization of the Tianhe-2 supercomputer 15m

        Supercomputer and other high performance computing resources can be useful supplements to the BESIII computing resources for simulation productions and data analysis. The supercomputer Tianhe-2 has ranked the No.1 on the Top500 certificate list for the sixth consecutive times during the year 2013 to 2015. This paper will describe the deployment singularity containers as well as the integration of local job submission system on the Tianhe-2 supercomputer in BESIII.

        Speaker: Qiumei Ma (IHEP China)
      • 15:45
        BESIII drift chamber tracking with machine learning 15m

        Drift chamber is the main tracking detector for high energy physics experiment like BESIII. Deep learning developments in the last few years have shown tremendous improvements in the analysis of data especially for object classification and parameter regression. Here we present a first study of deep learning architectures applied to BESIII Monte-carlo data to make estimation of the track parameter based on hit measurement of drift chamber.

        Speaker: Yao Zhang
      • 16:00
        BSM: Bundled Software Manager toolkit and the application for CEPC 15m

        Circular Electron Positron Collider (CEPC) is designed as a future Higgs Factory. Like other high energy physics experiment, the offline software consists of many packages. BSM (Bundled Software Manager) is thus created in order to simplify the deployment and usage of software which has many packages and dependencies.

        BSM utilizes git as the software repository. Different software versions are distinguished by git tags. The details of software are defined in the git repository including installation instructions, environment, dependencies, etc. Commands are supported for various shells, including bash, csh, zsh, tcsh and more could be extended. Json output and python API are also available for advanced development. The installation of each package could be configured separately and extended with customized handler. BSM manages the environment variables and the version cleaning and switching are easy. It also has fine environment control on a single package. Users can also define their own packages easily and these packages will be managed by BSM with simple configuration.

        CEPCSoft has already set up the deployment procedure with BSM. And BSM is also designed with flexibility to create different applications other than CEPCSoft. It is suitable for the projects including a lot of packages and it is safe for different BSM applications to coexist with each other under proper configuration.

        Speaker: Xianghu Zhao (Chinese Academy of Sciences (CN))
      • 16:15
        A fully unprivileged CernVM-FS 15m

        The CernVM File System provides the software and container distribution backbone for most High Energy and Nuclear Physics experiments. It is implemented as a file system in user-space (fuse) module, which permits its execution without any elevated privileges. Yet, mounting the file system in the first place is handled by a privileged suid helper program that is installed by the fuse package on most systems. The privileged nature of the mount system call is a serious hindrance to running CernVM-FS on opportunistic resource and supercomputers. Fortunately, recent developments in the Linux kernel and in the fuse user-space libraries enabled fully unprivileged mounting for fuse file systems (as of RHEL 8), or at least outsourcing the privileged mount system call to a custom, external process. This opens the door to several, very appealing new ways to use CernVM-FS, such as a generally usable "super pilot" consisting of the pilot code bundled with Singularity and CernVM-FS, or the on-demand instantiation of unprivileged, ephemeral containers to publish new CernVM-FS content from anywhere. In this contribution, we discuss the integration of these new Linux features with CernVM-FS and show some of its most promising, new applications.

        Speaker: Jakob Blomer (CERN)
      • 16:15
        A multi-purpose user interface for the iFDAQ of the COMPASS experiment 15m

        In HEP experiments, remote access to control systems is one of the fundamental pillars of efficient operations. At the same time, development of user interfaces with emphasis on usability can be one of the most labor-intensive software tasks to be undertaken in the life cycle of an experiment. While desirable, the development and maintenance of a large variety of interfaces (e.g., desktop control interface, web monitoring interface, development API...) is often simply not feasible, as far as manpower is concerned. We present a solution employed in the control software of the iFDAQ of the COMPASS experiment at CERN. Being a mix of a command-line and terminal tool, this interface can fulfill the roles of a dynamic monitoring interface, a control interface, and a scripting API simultaneously. Furthermore, it can easily be used as a remote access tool for operations experts, needing nearly no setup user-side and being compatible with smartphones. We also discuss the methodology and results of a concrete use case - automated run control for performance tests of the iFDAQ readout software.

        Speaker: Mr Antonin Kveton (Charles University (CZ))
      • 16:15
        A new PyROOT: Modern, Interoperable and more Pythonic 15m

        PyROOT is the name of ROOT’s automatic Python bindings, which allow to access all the ROOT functionality implemented in C++ from Python. Thanks to the ROOT type system and the Cling C++ interpreter, PyROOT creates Python proxies for C++ entities on the fly, thus avoiding to generate static bindings beforehand.

        PyROOT has been enhanced and modernised to meet the demands of the HEP Python community. On the one hand, it has been redesigned on top of the new Cppyy library, in order to benefit from the modern C++ features supported by the latter. On the other hand, PyROOT is now interoperable with other tools from the Python data science ecosystem, such as NumPy and pandas, being able to expose ROOT data to those tools and vice-versa. Moreover, PyROOT improved on customizing Python language features for C++ objects to blend in seamlessly in the Python ecosystem.

        Speaker: Enric Tejedor Saavedra (CERN)
      • 16:15
        Accelerating the simulation process in gas based charged particle detectors 15m

        Simulation is an important tool in the R&D process of detectors and their optimization. Fine tuning of detector parameters and running conditions can be achieved by means of advanced simulation tools thus reducing cost associated to prototyping.
        This simulation, however, in complex detector geometries, large volumes and high gas gain becomes computationally expensive and can run for several days or weeks.
        A common tool in charged particle detector simulation is the Garfield simulation suite. Although Garfield has been very critical to the simulation of many Micro-Pattern Gas Detectors (MPGD) variants, the simulation of recent complex structures such as (multi-stage GEM and Micromegas) is CPU consuming, in particular in situations where a high gas gain and large volumes are needed.

        In this work, we have explored the possibility to reduce the computation time by following several approaches. On one hand, we developed a parallel variant of the software allowing the simulation to be spread over hundreds of cores in a multicore distributed environment. Secondly, with advanced profiling, we optimized the electron avalanche development leading a speedup factor of 20 for large structures such as triple-GEM detectors. Moreover, we also adapted the code to multi-threading architecture allowing more computational efficiency.

        We will present the three strategies and the results in terms of speedup factor and parallel efficiency. Results from serial and parallel/optimized versions of the codes will also be presented and discussed.

        Speaker: Othmane Bouhali (Texas A & M University (US))
      • 16:15
        Alignment of the BESIII end cap TOF system 15m

        The end cap time-of-flight (ETOF) at Beijing Spectrometer (BESIII) was upgraded with multi-gap resistive plate chamber technology in order to improve the particle identification capability. The accurate knowledge of the detector real misalignment is important for getting close to the designed time resolution and the expected reconstruction efficiency of the end cap time-of-flight system. The incident position of charged track is extrapolated from multilayer drift chamber to the end cap time-of-flight detector based on the ideal geometry. The raw measured time difference between the two readout channels of one strip could be used to derive an incident position of the charged track using the transmission velocity of the signal in the readout strip, this position is obtained from the response of the detector, without any geometric hypothesis. A software is developed using the comparison between these two positions to provide the spatial position difference between the real detector and the ideal location. The procedure developed towards the alignment and the geometry is updated based on the result of the alignment. The time resolution and reconstruction achieved in recent years are also described in this talk.

        Speaker: Shengsen Sun (Institute of High Energy Physics)
      • 16:15
        An automatic solution to make HTCondor more stable and easier 15m

        HTCondor, with high scheduling performance, has been widely adopted for HEP clusters. Unlike other schedulers, HTCondor provides loose management functions to the work-nodes. We developed a Maintenance Automation Tool acronym as “HTCondor MAT“, focusing on resource management dynamically and error handing automatically.
        A central database is used to record various attributes of all computing resource and experiment requirements. Each worknode is configured by MAT automatically. The worknode status is collected and analyzed in real time. If the result shows error happened, the worknode would be reconfigured at once to avoid the bad effect to jobs. A smart wrapper script deployed at worknode monitors each job running period.
        HTCondor MAT has been deployed to the IHEP HTC cluster, which has more than 14,000 CPU cores. It decreases routine maintenance work of admin and anomaly happen to the cluster could be evicted automatically in time.

        Speaker: Mr Xiaowei Jiang (IHEP)
      • 16:15
        Analyzing storage access data with Apache-Spark and Jupiter notebooks 15m

        Running a data center is never a trivial job. In addition to daily
        routine tasks, service operation teams have to provide a meaningful
        information for monitoring, reporting and access pattern analytic.
        The dCache production instances at DESY, produce gigabytes of billing
        files per day. However, with a help of modern BigData analysis tools
        like Apache-Spark and Jupiter notebooks such task can be easily achieved.
        Moreover, the tool set for storage access analysts can be shared with
        scientific community making it re-usable computational resource as well
        as shared knowledge

        Speaker: Thomas Hartmann (Deutsches Elektronen-Synchrotron (DE))
      • 16:15
        Artificial load generation for xcache and reduction of application write delay 15m

        Two recent software development projects are described: The first is a framework for generating load for an xrootd based disk caching proxy (known as xcache) and verifying the generated data as delivered by the cache. The second is a service to reduce the effect of network latency on application execution time due to writing files to remote storage via the xrootd protocol. For both projects the problem, requirements and design considerations will be explained. The implementation will be covered.

        Speaker: Andrea Sciabà (CERN)
      • 16:15
        ATLAS Event Store and I/O developments in support for Production and Analysis in Run 3 15m

        During the long shutdown, ATLAS is preparing several fundamental changes to its offline event processing framework and analysis model. These include moving to multi-threaded reconstruction and simulation and reducing data duplication during derivation analysis by producing a combined mini-xAOD stream. These changes will allow ATLAS to take advantage of the higher luminosity at Run 3 without overstraining processing and storage capabilities. They also require significant changes to the underlying event store and the I/O framework to support them. These changes and their effects are discussed in the presentation:

        • The Run 2 I/O framework was overhauled to be thread-safe and
          minimize serial bottlenecks.
        • For object navigation, new immutable references are deployed, which
          don’t rely on storage container entry number so data can be merged
          in-memory.
        • Filter decisions can be used to annotate combined output stream
          allowing for fast event selection on input.
        • Compression algorithms and settings were optimized to allow efficient
          reading of event selections.
        Speaker: Marcin Nowak (Brookhaven National Laboratory (US))
      • 16:15
        ATLAS Level-1 Endcap Muon Trigger for Run 3 15m

        The LHC is expected to increase its center-of-mass energy to 14 GeV and an instantaneous luminosity to 2.4×1034 cm-2s-1 for Run-3 scheduled from 2021 to 2023. In order to cope with the high event rate, an upgrade of the ATLAS trigger system is required.
        The level-1 Endcap Muon trigger system identifies muons with high transverse momentum by combining data from a fast muon trigger detector, TGC. In the ongoing Phase-I upgrade, a new detector called the New-Small-Wheel (NSW) and RPC-BIS78, will be installed in the inner station region for the endcap muon trigger. Finer track information from the NSW and RPC-BIS78 can be used as part of the muon trigger logic to enhance performance significantly.
        In order to handle data from both TGC and NSW, some new electronics have been developed, including the trigger processor board known as Sector Logic (SL). The SL board has a modern FPGA to make use of Multi-Gigabit transceiver technology, which will be used to receive data from the NSW. The readout system for trigger data has also been re-designed, with the data transfer implemented with TCP/IP instead of a dedicated ASIC. This makes it possible to minimise the use of custom readout electronics and instead use some commercial PCs and network switches to collect, format and send the data. This presentation describes the aforementioned upgrades of the level-1 Endcap Muon trigger system. Particular emphasis will be placed on the new algorithm in Sector Logic. The expected trigger performance will also be discussed.

        Speaker: Atsushi Mizukami (High Energy Accelerator Research Organization (JP))
      • 16:15
        ATLAS Tile Calorimeter Conditions Database architecture and operations in Run-2 15m

        An overview of the Conditions Database (DB) structure for the hadronic Tile Calorimeter (TileCal), one of the ATLAS Detector sub-systems, is presented. ATLAS Conditions DB stores the data on the ORACLE backend, and the design and implementation has been developed using COOL (Conditions Objects for LCG) software package as a common persistency solution for the storage and management of the conditions data. TileCal Conditions and calibration data are stored in 4 separate Databases also known as schemas: TileCal Online and Offline DBs for data, DB for Monte Carlo (MC) simulation and Detector Control System (DCS) DB. In order to support the smooth TileCal operations during data taking period, experts perform the necessary calibrations, add the changes of detector status and other conditions data, prepare new conditions for data reprocessing and MC production campaigns, and upload the new up-to-date information into DB using the custom-made software tools. The procedure of TileCal conditions preparation, validation and uploading to DB is described, and some DB-related statistics collected in Run-2 is provided.

        Speaker: Iouri Smirnov (Northern Illinois University (US))
      • 16:15
        BNL Cloud Storage Service BNLBox 15m

        Large scientific data centers have recently begun providing a number of different types of data storage, to satisfy the various needs of their users. Users with interactive accounts, for example, might want a posix interface for easy access to the data from their interactive machines. Grid computing sites, on the other hand, likely need to provide an X509 based storage protocol, like SRM and GridFTP, since the data management system is built on top of them. Meanwhile, an experiment producing large amounts of data typically demands a service that provides archival storage for the safe keeping of their unique data. To access these various types of data, users must use specific sets of commands tailored for the respective storage, making access to their data complex and difficult. BNLBox is an attempt to provide a unified and easy to use storage service for all BNL users, scientists and engineers, to store their important documents, code and data. It is a cloud storage system with an intuitive web interface for novice users. It provides an automated synchronization feature that enables users to upload data to their cloud storage without manual intervention, freeing them to focus on analysis rather than data management software. It provides a posix interface for local interactive users, which simplifies data access from batch jobs as well. At the same time, it also provides users with a straightforward mechanism for archiving large data sets for later processing. The storage space can be used for both code and data within the compute job environment. This presentation will cover the various aspects of the BNLBox storage service.

        Speaker: Ofer Rind
      • 16:15
        CConcurrent Conditions Access across Validity Intervals in CMSSW 15m

        The CMS software system, known as CMSSW, has a generalized conditions, calibration, and geometry data products system called the EventSetup. The EventSetup caches results of reading or calculating data products based on the 'interval of validity', IOV, which is based on the time period for which that data product is appropriate. With the original single threaded CMSSW framework, updating only on an IOV boundary meant we only required memory for a single data product of a given type at any time during the program execution. In 2016 CMS transitioned to using a multi-threaded framework as a way to save on memory during processing. This was accomplished by amortizing the memory cost of EventSetup data products across multiple concurrent events. To initially accomplish that goal required synchronizing event processing across IOV boundaries, thereby decreasing the scalability of the system. In this presentation we will explain how we used 'limited concurrent task queues' to allow concurrent IOVs while still being able to limit the memory utilized. In addition, we will present performance measurements for both threading scalability of event throughtput and memory utilization.

        Speaker: Christopher Jones (Fermi National Accelerator Lab. (US))
      • 16:15
        CERNBox as the hyper-converged storage space at CERN: integrating DFS use-cases and home directories 15m

        CERN IT is reviewing its portfolio of applications, with the aim to incorporate open-source solutions wherever possible. In particular, the Windows-centric DFS file system is replaced by CERNBox for certain use-cases.

        Access to storage from Windows managed devices for end-users is largely covered by synchronization clients. However, online access using standard CIFS/SMB protocol is required in certain cases such as central login services (Terminal Services) and visitor desktop computers. We present recent work to introduce a set of Samba gateways running in High Availability cluster mode to enable direct access to the backend storage (EOS). This work covers all phases of the project: from a first prototype to a fully monitored service that can be scaled out according to needs, integrated in a central storage platform for the CERN users community, with an ever-growing user base of CERNBox that is now beyond 16K users.

        We will also describe the testing infrastructure that was put in place to benchmark and validate the protocol stack, emulating Windows application workloads such as Office suite and to support workflows with mixed online and sync traffic. In addition we will also reference technical components necessary for a successful integration of CIFS/SMB protocol, including the mapping of the Windows permissions model to EOS ACLs.

        In this paper we also describe the ongoing effort to migrate DFS home directories to this new infrastructure.

        Speaker: Luca Mascetti (CERN)
      • 16:15
        ComputeOps: container for High Performance Computing 15m

        The High Performance Computing (HPC) domain aims to optimize code in order to use the last multicore and parallel technologies including specific processor instructions. In this computing framework, portability and reproducibility are key concepts. A way to handle these requirements is to use Linux containers. These "light virtual machines" allow to encapsulate applications within its environment in Linux processes. Containers has been recently rediscovered due to their abilities to provide both multi-infrastructure environnement for developers and system administrators and reproducibility due to image building file. Two container solutions are emerging: Docker for micro-services and Singularity for computing applications. We present here the status of the ComputeOps project which has the goal to study the benefit of containers for HPC applications.

        Speaker: Sébastien Gadrat (CC-IN2P3)
      • 16:15
        Consolidating the Grid and interactive analysis infrastructure at DESY - past and future 15m

        DESY manages not only one of the largest Tier-2 sites with about 18 500 CPU cores for Grid workloads but also about 8000 CPU cores for interactive user analyses. In this presentation, we recapitulate the consolidation of the batch systems in a common HTCondor based setup and the lessons learned as both use cases differ in their goals. Followingly, we will give an outlook
        on the future developments.\newline
        While for Grid jobs startup latencies are negligible and the primary focus is on an optimal utilization of the resources, users of the {\it National Analysis Factory} for interactive analyses prefer a high responsiveness of the batch system as well as the storage.\newline
        In the ongoing evolution of the batch system, we are exploring two different approaches to abstract the batch node's host OS from the actual job OS. For Grid jobs we are running legacy workloads in lightweight Singularity containers deployed via CVMFS. For interactive NAF jobs, we move towards Docker containers for a more heavy weight replication of a full batch node to provide the users with their full set of accustomed tools.\newline
        To utilize resources best we further investigate the opportunistically backfilling resources, especially GPU nodes, between pools without interfering with user experiences.

        Speaker: Thomas Hartmann (Deutsches Elektronen-Synchrotron (DE))
      • 16:15
        Cosmos : A Unified Accounting System both for HTCondor and Slurm Clusters at IHEP 15m

        HTCondor is adopted to manage the High Throughput Computing (HTC) cluster at IHEP since 2017. Two months later in the same year, a Slurm cluster is set up to run High Performance Computing (HPC) jobs. To provide accounting service both for HTCondor and Slurm clusters, a unified accounting system named Cosmos is necessary to develop.
        However, different job workload brings different accounting requirements. Most jobs from the HTCondor cluster are single-core jobs, and more than 30 million jobs are submitted each year. In addition to these jobs, a legacy HTCondor Virtual Machine (VM) cluster is treated as the second HTCondor pool, which means that VM jobs have to be accounted as well. On the other side, most jobs run in the Slurm cluster are parallelism jobs, and some jobs are run on GPU worker nodes to accelerate computing. Besides, some qualified HTC jobs are going to migrate from the HTCondor cluster to the Slurm cluster for research purpose.
        To satisfy all the mentioned requirements, Cosmos is designed as a four-layer system, and layers from bottom to top are: data acquisition layer, data integration layer, data statistics layer and data presentation layer. In this proceeding, we present the background and development requirements of Cosmos system, the four-layer system architecture, the issues of each layer, and technical solutions to these issues. Cosmos is running as a production system for more than one year, and running results show that it’s a well-functioning system and fulfills requirements from both HTCondor and Slurm clusters.

        Speaker: Xiaowei Jiang (IHEP(中国科学院高能物理研究所))
      • 16:15
        Cyber security monitoring for IHEP data centers 15m

        In recent years, along with the rapid development of large scientific facilities and e-science worldwide, various cyber security threats have becoming a noticeable challenge in many data centers for scientific research, such as DDoS attack, ransomware, crypto-currency mining, data leak, etc.

        Intrusion and abnormality detection by collecting and analyzing security data is an important measure for enhancing the sensitivity of security status perception, level of security protection, and agility of security incident response. However, as the scale of data center growing, it's difficult to use a single security box to process the large volume of various data generated by network traffic, device and host logs, threat intelligence, and so on.

        In high energy physics (HEP) community, people are trying to establish a security operation center (SOC) for handle this problem. We are also trying to build a cyber security monitoring and analysis framework at Institute of High Energy Physics (IHEP), Chinese Academy of Sciences. At IHEP, we have four data centers, located in three different cities in China, the largest one has 4x10 Gbps IPv4 and IPv6 dual-stacked internet connection, and 2x80 Gbps inner data center network. It's really a challenge for us to handle the security related data generated by such a set of information assets.

        In this framework, Malware Information Sharing Platform (MISP) is deployed for threat intelligence exchanging with collaborated HEP institutes and universities. Network traffic is collected from switches and firewalls flows to a Zeek instance for traffic analysis. All the security data like Zeek logs, hosts/web logs, security device logs, along with vulnerability scanning results and assets detection results, etc., are are collected by Flume/Logstash/Syslog to a data pipeline named Kafka cluster. In this cluster, there are some Spark jobs running for stream processing, which are aimed at rapid intrusion and abnormality detection as well as data correlation and enrichment. Then all the processed data are written to Elasticsearch, MySQL and InfluxDB, and then visualized by Kibana and Grafana. Moreover, we also deployed a commercial SOC product for cross-checking, and we imported the threat intelligence data from the cloud of the product provider.

        Speaker: Qi Fazhi (IHEP)
      • 16:15
        Data-centric Graphical User Interface of the ATLAS EventIndex Service 15m

        ATLAS EventIndex Service keeps references to all real and simulated ATLAS events. Hadoop Mapfiles and HBase tables are used to store the Event Index data, a subset of data is also stored in the Oracle database. Several user interfaces are currently used to access and search the data. From the simple command line interface, through programmatical API to sophisticated Graphical Web Services. The presentation will describe the most recent, highly interactive Web Service. It provides a dynamic graph-like overview of all available data and data collections. Data are shown together with their relations, ownership, containment or overlaps. Each data entity then gives users a set of actions which are available for the referenced data. Some actions are provided directly by the Event Index system, others are interfaces to various ATLAS services. In many cases, specialised views are offered for detailed data inspection, like several kinds of histograms or Venn diagrams.
        The presentation will document the current status of the service, its possibilities and performance. The future system evolution as an front-end to the new Event Index Architecture based on the Apache Phoenix will be also described as well as possible extension to more general framework for giving a new, more intuitive access to all experiment data.

        Speaker: Julius Hrivnac (Centre National de la Recherche Scientifique (FR))
      • 16:15
        Dataset Searching Webapp in Belle II 15m

        Belle II is a global collaboration with over 700 physicists from 113 institutes. In order to fuel the physics analyses, a distributed grid of computing clusters consisting of tens of thousands of CPU-cores will house the multiple petabytes of data that will come out of the detector in years to come. However, the task of easily finding the particular datasets of interest to physicists with specific needs is challenging, since the quantities of data are so large and the data are distributed in computer centers around the world.
        Our solution to this problem is to tag data-files and datasets with metadata that describe their nature and purpose, which can then be searched for via a custom built web-based application hosted by the BelleDIRAC platform. For this to see widespread use, it must be able to take in a set of user specified search criteria and return a list of all matching datasets. It must also be capable of taking in the location of a dataset and return a list of all its associated metadata back to the user. All of this should be done with an intuitive user interface and return results in an acceptably short time frame. The front-end is built with ExtJS while the back end-and table manipulation is handled with Python and MySQL.

        Speaker: Kim Smith (University of Melbourne)
      • 16:15
        Debugging Compute Clusters with Techniques from Functional Programming and Text Stream Processing 15m

        Monitoring is an indispensable tool for the operation of any
        large installment of grid or cluster computing, be it high
        energy physics or elsewhere. Usually, monitoring is configured
        to collect a small amount of data, just enough to enable
        detection of abnormal conditions. Once detected, the abnormal
        condition is handled by gathering all information from the
        affected components. This data is processed by querying it in a
        manner similar to a database.

        This contribution shows how the metaphor of a debugger (for
        software applications) can be transferred to a compute cluster. The concepts of variables, assertions and breakpoints that are
        used in debugging can be applied to monitoring by defining
        variables as the quantities recorded by monitoring and
        breakpoints as invariants formulated through these variables. It
        is found that embedding fragments of a data extracting and
        reporting tool such as the UNIX tool awk facilitates concise
        notations for commonly used variables since tools like awk are
        designed to process large event streams (in textual
        representations) with bounded memory. Additionally, it is found
        that a functional notation similar to both the pipe notation
        used in the UNIX shell and the pointfree style used in
        functional programming simplifies the combining of variables
        that commonly occur when formulating breakpoints.

        Speaker: Mr Alexander Adler (Johann-Wolfgang-Goethe Univ. (DE))
      • 16:15
        Distributed Computing for the Project 8 experiment 15m

        The Project 8 collaboration aims to measure the absolute neutrino mass or improve on the current limit by measuring the tritium beta decay electron spectrum. We present the current distributed computing model for the Project 8 experiment and requirements for future phases. Project 8 is in its second phase of data taking with a near continuous data rate of 1Gbps. The current computing model uses DIRAC (Distributed Infrastructure with Remote Agent Control) for its workflow and data management. A detailed meta-data assignment using the DIRAC File-Catalog is used to automate raw data transfers and subsequent stages of data processing. The DIRAC system is deployed on containers managed using a Kubernetes cluster to provide a scalable infrastructure. A modified DIRAC Site Director provides the ability to submit jobs using singularity on opportunistic High-Performance Computing sites.

        Speaker: Benjamin LaRoque
      • 16:15
        Distributed resources of the Czech WLCG Tier-2 center 15m

        The Computing Center of the Institute of Physics (CC FZU) of the Czech
        Academy of Sciences provides compute and storage capacity to several
        physics experiments. Most resources are used by two LHC experiments,
        ALICE and ATLAS. In the WLCG, which coordinates computing activities for
        the LHC experiments, the computing center is a Tier-2. The rest of
        computing resources is used by astroparticle experiments Pierre
        Auger Observatory (PAO) and Cherenkov Telescope Array (CTA) and
        particle experiments like NOvA and DUNE.
        Storage capacity is distributed to several locations. DPM servers used
        by the ATLAS and the PAO are all in the same server room. ALICE uses
        several xrootd servers located at the Nuclear Physics Institute in Rez,
        about 10 km away. The storage capacity for the ATLAS and the PAO is
        extended by resources of the CESNET (the Czech National Grid Initiative
        representative) located in Ostrava, more than 100 km away from the CC
        FZU. Storage is managed by a dCache instance, which is published in the CC
        FZU BDII. ATLAS users can use these resources using the standard ATLAS
        tools in the same way as the local storage without noticing this
        geographical distribution.
        The computing center provides about 8k CPU cores which are used by the
        experiments based on fair-share. The CPUs are distributed amongst server
        rooms in the Institute of Physics, in the Faculty of Mathematics and
        Physics of the Charles University, and at CESNET. For the ATLAS
        experiment, the resources are extended by opportunistic usage of the
        Salomon HPC provided by the Czech national HPC center IT4Innovations in
        Ostrava. The HPC provides 24-core nodes. The maximum number of allowed
        single-node jobs in the batch system is 200. The contribution of the HPC
        to the CPU consumption by the ATLAS experiment is about 15% on average.

        Speaker: Jiri Chudoba (Acad. of Sciences of the Czech Rep. (CZ))
      • 16:15
        Event Streaming Service for ATLAS Event Processing 15m

        The ATLAS Event Streaming Service (ESS) is an approach to preprocess and deliver data for Event Service (ES) that has implemented a fine-grained approach for ATLAS event processing. The ESS allows one to asynchronously deliver only the input events required by ES processing, with the aim to decrease data traffic over WAN and improve overall data processing throughput. A prototype of ESS is developed to deliver streaming events to fine-grained ES jobs. Based on it, an Intelligent Data Delivery Service (IDDS) is under development to separate the “cold format” and the processing format of the data, which opens the opportunity to include the production systems of other HEP experiments.
        Here we will present the ESS architecture, features and capabilities, and its applications for the ATLAS Event Service. We will also describe the motivations for IDDS system and its advantages.

        Speaker: Brian Paul Bockelman (University of Nebraska Lincoln (US))
      • 16:15
        Evolution of the CloudVeneto.it private cloud to support research and innovation 15m

        CloudVeneto.it was initially funded and deployed by INFN in 2014 for serving the computational and storage demands of INFN research projects mainly related to HEP and Nuclear Physics. It is an OpenStack-based scientific cloud with resources spread across two different sites connected with a high speed optical link: the INFN Padova Unit and the INFN Legnaro National Laboratories. The infrastructure has grown throughout the years with additional funds from ten University of Padova departments, and nowadays supports a broader range of scientific and engineering disciplines. Its hardware resources provide around 2000 computational cores and 300 TB of storage to about 250 users working for more than 70 projects. In the last months we enhanced the cloud platform in two ways: 1) by integrating a number of heterogeneous GPU cards to address the special needs of user communities whose computations involve machine learning training; 2) by enabling the users to simply deploy on-demand Kubernetes clusters for Big Data analytics applications taking advantage of the operator framework. In particular, the Kubernetes operators for Apache Kafka and Spark platforms were integrated to address real-time data ingestion and streaming processing on the cloud. The article will describe the technical details of the these two solutions and their integration with the cloud infrastructure.

        Speaker: Marco Zanetti (Universita e INFN, Padova (IT))
      • 16:15
        Evolution of the filesystem interface of the EOS Open Storage system 15m

        With the ongoing decomissioning of the AFS filesystem at CERN many use cases have been migrated to the EOS storage system at CERN.

        To cope with additional requirements the filesystem interface implemented using FUSE had been rewritten since 2017. The new implementation supports strong security in conventional, VM and container environments. It is in production for the CERNBOX EOS service since beginning of 2019 with continous improvements in stability, operability and usability, connecting tens of thousands of filesystem clients to the CERNBOX/EOSHOME service.

        In this paper we will discuss the challenges, current status, limitations and future improvements of the EOS filesystem interface.

        Speaker: Andreas Joachim Peters (CERN)
      • 16:15
        Evolution of the STFC’s Tape archival service 15m

        The STFC CASTOR tape service is responsible for the management of over 80PB of data including 45PB generated by the LHC experiments for the RAL Tier-1. In the last few years there have been several disruptive changes that have or are necessitating significant changes to the service. At the end of 2016, Oracle, which provided the tape libraries, drives and media announced they were leaving the tape market. In 2017, the Echo (Tier-1 disk) storage service entered production and disk only storage migrated away from CASTOR. In 2017, CERN, which provides support for CASTOR, started to test their replacement to CASTOR called CTA.

        Since October 2018, a new shared CASTOR instance has been in production. This instance is a major simplification from the previous four. In this paper we describe the setup and performance of this instance which includes two sets of failure-tolerant management nodes that ensure improved reliability and a single unified tape cache that has displayed increased access rates to tape data compared to previous separate tape cache pools.

        In March 2019, a new Spectra Logic Tape robot was delivered to RAL. This uses both LTO and IBM media. This paper will describe the tests that were carried out on this system, which includes multiple sets of dense and sparse tape reads to assess the throughput performance of the library for various use cases.

        Finally, this paper will describe the ongoing work exploring possible new, non-SRM tape management systems that will eventually replace CASTOR.

        Speaker: Katy Ellis (Science and Technology Facilities Council STFC (GB))
      • 16:15
        Fast and Efficient Entropy Compression of ALICE Data using ANS Coding 15m

        With the beginning of LHC Run 3, the upgraded ALICE detector will record Pb-Pb collisions at an interaction rate of 50 kHz using continuous readout, resulting in raw data rates of over 3.5TB/s marking a hundredfold increase over Run 2. Since permanent storage at this rate is unfeasible and exceeds available capacities, a sequence of highly effective compression and data reduction steps is required. Most of these steps perform lossy compression based on techniques like zero suppression, track finding and clusterization without affecting physics. The final compression step is the lossless entropy encoding.
        Huffman coding as used for entropy coding in Run 2 is fast and simple, but suffers from an inefficiency of up to one bit per encoded symbol compared to the entropy, leaving room for optimization.

        For Run 3 entropy coding based on asymmetric numeral systems (ANS) is under evaluation. ANS achieves compression by encoding source symbols into a single, infinite precision integer state variable using arithmetic operations based on a symbols probability, thus overcoming the one bit inefficiency of Huffman coders. This allows for compression ratios very close to the entropy. Furthermore the mathematical properties of ANS allow efficient parallelization using vector units on CPUs or GPUs.
        This contribution describes a custom implementation of an ANS encoder and decoder required to cope with the various distributions of source data that have to be taken into account to achieve the desired high compression ratios for the ALICE detectors - in particular the Time Projection Chamber (TPC) and the Inner Tracking System (ITS). To ensure encoding bandwidth is on pair with the high data rates, our implementation leverages SIMD vector units of CPUS and on-GPU data compression is under evaluation.

        Speaker: Michael Lettrich (Technische Universität Muenchen (DE))
      • 16:15
        Fast Inference for Machine Learning in ROOT/TMVA 15m

        ROOT provides, through TMVA, machine learning tools for data analysis at HEP experiments and beyond. However, with the rapidly evolving ecosystem for machine learning, the focus of TMVA is shifting.

        In this poster, we present the new developments and strategy of TMVA, which will allow the analyst to integrate seamlessly, and effectively, different workflows in the diversified machine-learning landscape. Focus is put on a fast machine learning inference system, which will enable analysts to deploy their machine learning models rapidly on large scale datasets. We present the technical details of a fast inference system for decision tree algorithms, included in the next ROOT release (6.20). We further present development status and proposal for a fast inference interface and code generator for ONNX-based Deep Learning models.

        Speaker: Sitong An (CERN, Carnegie Mellon University (US))
      • 16:15
        Fast Simulation in ATLAS 15m

        The ATLAS physics program relies on very large samples of simulated events. Most of these samples are produced with GEANT4, which provides a highly detailed and accurate simulation of the ATLAS detector. However, this accuracy comes with a high price in CPU, and the sensitivity of many physics analysis is already limited by the available Monte Carlo statistics and will be even more so in the future as datasets grow. To solve this problem, sophisticated fast simulation tools are developed and they will become the default tools in ATLAS production in Run-3 and beyond.
        The slowest component is the simulation of the calorimeter showers. Those are replaced by a new parametrised description of the longitudinal and lateral energy deposits, including machine learning approaches, achieving a fast but accurate description. Other fast simulation tools replace the inner detector
        simulation, as well as digitization and reconstruction algorithms, achieving up to two orders of magnitude improvement in speed.
        In this talk we will describe the new tools for fast simulation that have been developed by ATLAS, review their technical and physics performance, and demonstrate their potential to transform physics analyses.

        Speaker: Heather Gray (LBNL)
      • 16:15
        Feasibility tests of RoCE for the cluster-based event building in LHCb 15m

        This paper evaluates the utilization of RDMA over Converged Ethernet (RoCE) for the Run3 LHCb event building at CERN. The acquisition system of the detector will collect partial data from approximately 1000 separate detector streams. Total estimated throughput equals 40 terabits per second. Full events will be assembled for subsequent processing and data selection in the filtering farm of the online trigger. As a result, inter-node large-throughput transmissions with a combination of 100 and 25 Gigabit-per-second will be essential features of the system. Therefore, the data exchange mechanism of the cluster must utilize memory-lightweight data transmission protocols.
        In this work, the RoCE high-throughput kernel bypass Ethernet-based protocol is benchmarked as an applicable technology for the event building network. CPU and memory bandwidth utilization for RoCE-based data transmissions is investigated and discussed. A comparison of RoCE with InfiniBand protocol is presented. Preliminary performance results are discussed with the selected network hardware supporting the protocol. Relevant utilization and interoperability issues are detailed along with lessons learned along the road.

        Speaker: Rafal Dominik Krawczyk (CERN)
      • 16:15
        Generative modeling for shower simulation in ATLAS 15m

        Modeling the physics of a detector's response to particle collisions is one of the most CPU intensive and time consuming aspects of LHC computing. With the upcoming high-luminosity upgrade and the need to have even larger simulated datasets to support physics analysis, the development of new faster simulation techniques but with sufficiently accurate physics performance is required. The current ATLAS fast calorimeter simulation technique, based on parametrizations of the calorimeter response in the longitudinal and transverse direction given a single particle’s eta and energy, is being updated to higher accuracy including machine learning approaches. Here we report on a prototype using cutting edge machine learning approaches to learn the appropriate detector output response, which are expected to lead to an improved modelling of correlations within showers. The model is trained on a set of fully simulated events and the goal is to generate new outputs. We are studying Variational Auto-Encoders (VAEs) and Generative Adversarial Networks (GANs) to model particle showers in the ATLAS calorimeter. In this study we present an exploratory analysis of both models trained with different optimization tricks and criteria. Thus, our goal is get a deeper understanding of the learning process and how it leads to better improvement of the generation performance.

        Speaker: Johnny Raine (Universite de Geneve (CH))
      • 16:15
        Global fit of the minimal composite Higgs model using differential evolution 15m

        Composite Higgs models (CHMs), in which the Higgs boson is a bound state of an as-yet undetected strongly interacting sector, offer an attractive solution to the hierarchy problem while featuring rich particle phenomenology at the few-TeV scale. Of particular interest is the minimal CHM (MCHM), based on the $SO(5) \to SO(4)$ symmetry breaking pattern. However, the complexity of its parameter space has prevented a global fit of the model using simple parameter scans and even nested sampling and MCMC optimisation algorithms. We have instead taken a genetic optimisation approach using the differential evolution software Diver to produce the first convergent global fit of the two-site 4D MCHM with partially composite quarks, employing constraints from the electroweak and flavour sectors along with recent collider measurements. Here we will present preliminary results of the global fit with special attention paid to the fine-tuning of the model and the prospects of probing the MCHM in future collider searches.

        Speaker: Ethan Carragher (University of Adelaide)
      • 16:15
        HEP Data Query Challenges 15m

        Analysis languages must, first and foremost, carefully describe how to extract and aggregate data. All analysis languages must be able to make a plot of an event’s Missing Energy, for example. Of course, much more complex queries must also be supported, like making the plot of Missing Energy only for events with at least two jets that satisfy certain requirements. A project was started to try to collect examples in repo’s of 8 such challenges ranging in complexity from the simplest to the complex. This presentation will describe the effort and the list of languages that have participated, and highlight some examples from the repositories submitted at the time of CHEP. It should be noted that while the authors are helping to coordinate the effort, the language authors and others have done the work to implement the various challenges.

        Speaker: Mason Proffitt (University of Washington (US))
      • 16:15
        Hybrid analysis pipelines in the REANA reproducible analysis platform 15m

        In this paper we introduce and study the feasibility of running hybrid analysis pipelines using the REANA reproducible analysis platform. The REANA platform allows researchers to specify declarative computational workflow steps describing the analysis process and to execute the workflow pipelines on remote containerised Kubernetes-orchestrated compute clouds. We have designed an abstract job controller component permitting to execute different parts of the analysis workflow on different compute backends, such as HTCondor and SLURM. We have prototyped the designed solution including the job execution, job monitoring, and input/output file transfer mechanism between the various backends used in the computational workflow. We have tested the prototyped solution using several model particle physics analyses. The present work paves the way towards supporting hybrid analysis workflows in the REANA reusable analysis platform and studies the underlying reproducibility challenges inherent to using hybrid analysis patterns in particle physics data analyses.

        Speaker: Diego Rodriguez Rodriguez (CERN)
      • 16:15
        IRIS – Providing a Nationally Accessible Infrastructure for UK Science 15m

        In many countries around the world, the development of national infrastructures for science either has been implemented or are under serious consideration by governments and funding bodies. Current examples include ARDC in Australia, CANARIE in Canada and MTA Cloud in Hungary. These infrastructures provide access to compute and storage to a wide swathe of user communities and represent a collaboration between users, providers and, in some cases, industry to maximise the impact of the investments made.

        The UK has embarked on a project called IRIS to develop a sustainable e-infrastructure based on the needs of a diverse set of communities. Building on the success of the UK component of wLCG and the innovations made, a number of research institutes and universities are working with several research groups to co-design an infrastructure, including support services, which take this to a level applicable to a wider use base.

        We present the preparatory work leading to the definition of this infrastructure, showing the wide variety of use cases which require to be supported. This leads us to a definition of the hardware and interface requirements needed to meet this diverse set of criteria, and the support posts identified in order to make best use of this facility and sustain it into the future.

        Speaker: Andrew Lahiff
      • 16:15
        Knowledge sharing on deep learning in physics research using VISPA 15m

        The VISPA (VISual Physics Analysis) project provides a streamlined work environment for physics analyses and hands-on teaching experiences with a focus on deep learning.
        VISPA has already been successfully used in HEP analyses and teaching and is now being further developed into an interactive deep learning platform.
        One specific example is to meet knowledge sharing needs in deep learning by combining paper, code and data at a central place.
        Additionally the possibility to run it directly from the web browser is a key feature of this development.
        Any SSH reachable resource can be accessed via the VISPA web interface.
        This enables a flexible and experiment agnostic computing experience.
        The user interface is based on JupyterLab and is extended with analysis specific tools, such as a parametric file browser and TensorBoard.
        Our VISPA instance is backed by extensive GPU resources and a rich software environment.
        We present the current status of the VISPA project and its upcoming new features.

        Speaker: Manfred Peter Fackeldey (Rheinisch Westfaelische Tech. Hoch. (DE))
      • 16:15
        Making cheap disks cheap with dCache storage events and QoS 15m

        Within the DOMA working group, the QoS activity is looking at how best to describe innovative technologies and deployments. Once scenario that has emerged is providing storage that uses end-of-warranty disks: the cheap (almost free) nature of this storage is offset by a much larger likelihood of data loss. In some situations, this trade-off is acceptable, provided the operational overhead of handling this data loss is not excessive.

        In this paper, we present a model where dCache provides access to this data. Improvements within dCache administrative interface allow for almost no operational overhead for handling such storage. The storage events concept allows experiment data-management frameworks, such as Rucio, to learn of any data-loss in a robust and fully automated fashion. These frameworks can then follow strategies to recover from these problems; for example, by copying the lost data back into dCache.

        Speaker: Tigran Mkrtchyan (DESY)
      • 16:15
        Migration of user and project spaces with EOS\CERNBox: experience on scaling and large-scale operations 15m

        EOS is the key component of the CERN Storage strategy and is behind the success of CERNBox, the CERN cloud synchronisation service which allows syncing and sharing files on all major mobile and desktop platforms aiming to provide offline availability to any data stored in the infrastructure.

        CERNBox faced and enormous success within the CERN users' community thanks to its always increasing popularity and to its integration with a multitude of other CERN services such as Batch, SWAN, Microsoft Office to mention few.

        This success directly translated into an exponential growth in the last couple of years in terms of files and data stored which were leading to an explosion of the in-memory namespace technology used for the main catalogue.

        The original deployment from 2014 has proven effective and reliable, however the infrastructure was very quickly approaching the limits of its scalability and its capability of coping with the exponential growth. The sense of urgency was real: if the memory limit of the namespace server was reached, the service would simply stop. To counter this threat the storage team re-designed the EOS backend service architecture for end-user and project spaces.

        In this paper we present our new EOS deployment which entered in production during summer 2018 and we show how we successfully sharded the system into smaller failure domains and transform the original scale-up namespace deployment into a less expensive and more robust scale-out solution. We report about the experience gained so far in the simplification of the cluster upgrade procedures, now completely transparent, and the effort behind a transparent migration of 16k users from the old architecture into the new EOS architecture.

        Speaker: Luca Mascetti (CERN)
      • 16:15
        Monitoring and Analytics at INFN Tier-1: the next step 15m

        In modern data centers an effective and efficient monitoring system is a critical asset, yet a continuous concern for administrators. Since its birth, INFN Tier-1 data center, hosted at CNAF, has used various monitoring tools all replaced, since a few years, by a system common to all CNAF departments (based on Sensu, Influxdb, Grafana).
        Given the complexity of the inter-dependencies of the several services running at the data center and the foreseen large increase of resources in the near future, a more powerful and versatile monitoring system is needed. This new monitoring system should be able to automatically correlate log files and metrics coming from heterogeneous sources and devices (including services, hardware and infrastructure) thus providing us with a suitable framework to implement a solution for the predictive analysis of the status of the whole environment.
        In particular, the possibility to correlate IT infrastructure monitoring information with the logs of running applications is of great relevance in order to be able to quickly find application failure root cause. At the same time, a modern, flexible and user friendly analytics solution is needed in order to enable users, IT engineers and IT managers to extract valuable information from the different sources of collected data in a timely fashion. In this paper, a prototype of such a system, installed at the INFN Tier-1, is described with an assessment of the state and an evaluation of the resources needed for a fully production system. Technologies adopted, amount of foreseen data, target KPIs and production design is illustrated. Also, some aspects about possible usage of Machine Learning techniques, in order to automate the operations tasks, are covered with some examples.

        Speaker: Barbara Martelli (INFN CNAF)
      • 16:15
        Named Data Networking based File Access for XRootD 15m

        We present an NDN-based XRootD plugin and associated methods which have been built for data access in the CMS and other experiments at the LHC, its status and plans for ongoing development.

        Named Data Networking (NDN) is a leading Future Internet Architecture where data in the network is accessed directly by its name rather than the location of the host where it resides. NDN enables the joint design of multipath forwarding and caching to achieve superior latency and failover performance. The Caltech team together with Northeastern University and other collaborators in the SDN Assisted NDN for Data Intensive Experiments (SANDIE) project has implemented a corresponding NDN naming scheme in order to access datasets, where the naming structure follows the format /ndn/xrootd/<file system call>/<filename>/<segment no><additional info>, and an Open Storage System (OSS) plugin for XRootD.

        The XRootD plugin is a consumer entity that translates each filesystem call into an Interest Packet and sends it over the network using a local or remote NDN forwarder to find the corresponding data file from caches or producers. Alongside the plugin, a corresponding producer has been implemented as a service running on CentOS systems. The producer is capable of communicating with multiple file systems (HDFS, CEPH) and on receiving Interests it responds with data encapsulating a byte range at an offset from an existing file, specified in the name of the Interest packet, or an error code specifying the failure that occurred while trying to access it, e.g. the non-existence of a file containing the byte range requested.

        In this paper we present the architecture of the NDN-based OSS plugin for XRootD and its implementation details. In addition, first results of transfer tests and data access comparisons are presented along with details about the setup, and future plans for software, file management and platform development.

        Speaker: Jean-Roch Vlimant (California Institute of Technology (US))
      • 16:15
        Offline Software Management of the AMS experiment 15m

        The Alpha Magnetic Spectrometer (AMS) is a particle physics experiment installed and operating on board of the International Space Station (ISS) from May 2011 and expected to last through 2024 and beyond. The AMS offline software is used for data reconstruction, Monte-Carlo simulation and physics analysis. This paper presents how we manage the offline software, including the version control, the building, the documentation, the functional and performance testing, etc.

        Speaker: Jiahui Wei (Universite de Geneve (CH))
      • 16:15
        Particle Track Reconstruction with Quantum Algorithms 15m

        Accurate particle track reconstruction will be a major challenge for the High Luminosity LHC experiments. Increase in the expected number of simultaneous collisions and the high detector occupancy will make the algorithms extremely demanding in terms of time and computing resources.

        The sheer increase in the number of hits would increase the complexity exponentially, however the finite resolution of the detector and the physical “closeness” of the hits increase the ambiguity of their assignment to a trajectory, making the track reconstruction problem a major obstacle to the correct interpretation of the HL-LHC data. Most methods currently in use are based on the Kalman filter: they have shown to be robust and to provide good physics performance, however, they are expected to scale worse than quadratically.

        Designing an algorithm capable of reducing the combinatorial background at the hit level, would provide a much “cleaner” initial seed to the Kalman filter, thus strongly reducing the total processing time.

        One of the salient features of Quantum Computers is the ability to evaluate a very large number of states simultaneously, making them an ideal instrument for searches in a large parameter space and, in fact, different R&D initiatives are exploring how Quantum Tracking Algorithms could leverage such capabilities. In this paper we present our work on the implementation of a quantum-based track finding algorithm aimed at reducing combinatorial background during the initial seeding stage. We use the publicly available dataset designed for last year’s kaggle TrackML challenge.

        Speaker: Federico Carminati (CERN)
      • 16:15
        Predicting resource usage for enhanced job scheduling for opportunistic resources in HEP 15m

        Job schedulers in high energy physics require accurate information about predicted resource consumption of a job to assign jobs to the most reasonable, available resources. For example, job schedulers evaluate information about the runtime, numbers of requested cores, or size of memory, and disk space. Users, therefore, specify those information when submitting their jobs and workflows. Yet, the information provided by users cannot be considered entirely accurate. There are several reasons for this, including the heterogeneity of resources, regular changes to the underlying workflows, external dependencies or even lack of knowledge. This inaccuracy can result in inefficient utilisation of assigned resources by either blocking unused resources or exceeding reserved resources.

        With the increasing demand for the integration of opportunistic resources to extend the available WLCG computing resources, the accuracy of predicted resource consumption is of particular importance. Only an accurate prediction of resource consumption can enable a proper selection, allocation, and integration of resources to minimise the overall costs. We therefore propose to improve the indicated resource consumption of end-users with predictions to improve the resource utilisation of allocated opportunistic resources.
        In this contribution, we present our results and the impact of our prediction for both, end-user workflows and production workflows including pilot jobs. Our work focuses on resource consumption of CPU and memory but presents a generic approach that is ready for future use of other resources, such as GPUs.

        Speaker: Max Fischer (Karlsruhe Institute of Technology)
      • 16:15
        Quantum Optimization of Worldwide LHC Computing Grid data placement 15m

        The Worldwide LHC Computing Grid (WLCG) processes all LHC data and it has been the computing platform that has allowed the discovery of the Higgs Boson. Optimal usage of its resources represents a major challenge. Attempts at simulating this complex and highly non-linear environment did not yield practically usable results. For job submission and management, a satisfactory solution was implemented, based on a local optimisation of the workload, but optimal data placement remains a problem. Currently, it can be distributed according to some “reasonable” heuristics, such as topological distance between the data producer and the storage element.

        With the increase in the data produced by the HL-LHC experiments, a better data placement strategy becomes extremely relevant. While this problem has shown to be intractable till now, Quantum Computing can substantially improve the optimization of the storage.

        This contribution describes the development of a quantum algorithm for the optimisation of the WLCG storage distribution, starting from the specific case of the ALICE experiment. We intend to determine the optimal storage, movement and access of the data produced by the ALICE experiment in quasi real-time, in order to improve resource allocation and usage and to increase the efficiency of data handling workflow. This work is done in collaboration with Google, which is developing one of the most advanced hardware and software Quantum Computing programmes.

        Speaker: Federico Carminati (CERN)
      • 16:15
        ROOT I/O improvements for HEP analysis 15m

        We overview recent changes in the ROOT I/O system, increasing performance and enhancing it and improving its interaction with other data analysis ecosystems. Both the newly introduced compression algorithms, the much faster Bulk I/O data path, and a few additional techniques have the potential to significantly to improve experiments’ software performance.
        The need for efficient lossless data compression has grown significantly as the amount of HEP data collected, transmitted, and stored has dramatically increased during the LHC era. While compression reduces storage space and, potentially, I/O bandwidth usage, it should not be applied blindly: there are significant trade-offs between the increased CPU cost for reading and writing files and the reduce storage space.
        As ROOT I/O is responsible for providing serializing functionality for complex C++ objects, for example for physics reconstruction. At the same time, analysis workflows typically have simpler objects compared to reconstruction. This later case’s performance can be improve by using the ROOT “bulk I/O” interface, allowing to access multiple events to be returned per one library call.

        Speaker: Oksana Shadura (University of Nebraska Lincoln (US))
      • 16:15
        Security Mechanism for user access to Single SSID WLAN 15m

        Wireless local area network (WLAN) technology is widely used in various enterprises and institutions. In order to facilitate the use of users, they often provide a single ssid access point, resulting in different identities of users authenticated and authorized can connect to the wireless network anytime, anywhere as needed and obtain the same accessible network resources such as bandwidth, access control (ACL) and so on. Multiple ssid can solve the problem but it will be confused to users who don’t know which ssid can be connected. Although we could prevent visitors from accessing intranet resources by isolating the wireless network from the internal network, this would make it impossible for users to use the wireless network for internal office work. In this paper, we propose an access control sysytem that grouping users according to the different identities and users authenticated and authorized can access different network resources because a wireless access point dynamically maps an ssid provided by a mobile station to a BSSID based on a VLAN assignment. The deployment experiment of the solution proves that users of different identities accessing the same wireless network can set different access policies, which effectively improves the security of the wireless network and simplifies the management of the wireless network.

        Speaker: Li Wang (IHEP)
      • 16:15
        Space point calibration of the ALICE TPC with track residuals 15m

        In the upcoming LHC Run 3, starting in 2021, the upgraded Time Projection Chamber (TPC) of the ALICE experiment will record minimum bias Pb--Pb collisions in a continuous readout mode at 50 kHz interaction rate. This corresponds to typically 4-5 overlapping collisions in the detector. Despite careful tuning of the new quadruple GEM-based readout chambers, which fulfill the design requirement of an ion back flow below 1%, these conditions will lead to space charge distortions of several centimeter that fluctuate in time. They will be corrected via a calibration procedure that uses the information of the Inner Tracking System (ITS) and the Transition Radiation Detector (TRD) systems. They surround the TPC internally and externally, respectively. Such procedure is able to restore the TPCs intrinsic track resolution of a few hundred micrometer.

        We present the required online tracking algorithm for the TRD, which is based on a Kalman filter. The procedure matches extrapolated ITS-TPC tracks to TRD space points utilizing GPUs.
        Subsequently these global tracks are refitted neglecting the TPC information. The residuals of the TPC clusters to the interpolation of the refitted tracks are used to create a map of space charge distortions. Regular updates of the map compensate changes in the TPC conditions. The map is applied in the final reconstruction of the data. First performance results of the tracking algorithm and the space charge distortion maps will be shown.

        Speaker: Marten Ole Schmidt (Ruprecht Karls Universitaet Heidelberg (DE))
      • 16:15
        Status of the Belle II simulation library 15m

        The SuperKEKB collider and the Belle II experiment started Phase III at the beginning of 2019. The run is designed to collect a data sample of up to 50/ab at the collision energy of the Upsilon(4S) resonance for the next decade. The Belle II software library is created to ensure the accuracy and efficiency needed to
        accomodate this next generation B factory experiment.

        The central component of the Belle II simulation library is Geant4, which is one of the most popular software packages used by high energy experiments. In this talk, we will summarize the current status of the simulation library and will review the efforts to optimize the Geant4 functionalities in the Belle II specific environment. The discussed topics will include the optimization of the physics list and upgrade to Geant4 version 10.5.

        Speaker: Doris Yangsoo Kim (Soongsil University)
      • 16:15
        Striped Data Analysis Framework 15m

        Traditionally, High Energy data analysis is based on the model where data are stored in files and analyzed
        by running multiple analysis processes, each reading one or more of the data files. This process involves
        repeated data reduction step, which produces smaller files, which is time consuming and leads to data duplication. We propose an alternative approach to data storage and analysis, based on the Big Data
        technologies. The idea is to store each element of data once and only once in a distributed scalable
        database and analyze data by reading only "interesting" pieces of data from the database.

        To make this approach possible, we developed columnar Striped Data Representation Format as the basis of the
        framework. Traditional columnar approach allows for efficient analysis of complex data structures. While keeping all the benefits of columnar data representation, striped mechanism goes further by enabling efficient parallelization of computations and flexible distribution of data analysis.

        The framework includes scalable and elastic data storage, compute and user analysis backend components.
        The framework uses off-the shelf web services and data caching technologies as the compute/data co-location mechanism. Flexible architecture allows the framework run in the cloud using container technologies. The framework offers Python/Jupyter as the user analysis backend platform, but can also run in
        command line or batch mode.

        In the article, we will present the results of the FNAL-LDRD-2016-032 FNAL LDRD project, the design,
        implementation, features and performance characteristics of the Striped Data Analysis Framework.

        Speaker: Oliver Gutsche (Fermi National Accelerator Lab. (US))
      • 16:15
        Study of the influence of initial-state fluctuations on hydrodynamic simulations 15m

        In this work, we focus on assessing the contribution of the initial-state fluctuations of heavy ion collision in the hydrodynamic simulations. We try to answer the question of whether the hydrodynamic simulation retains the same level of fluctuation in the final-state as for the initial stage. In another scenario, the hydrodynamic simulations the fluctuation drowns in the final distribution of expanding matter. For this purpose, we prepared sufficient relativistic hydrodynamic program to study A+A interaction which allows analyzing initial-state fluctuations in the bulk nuclear matter. For such an assumption, it is better to use high spatial resolution. Therefore, we applied the (3+1) dimensional Cartesian coordinate system and we implemented our program using parallel computing on graphics cards processors Graphics Processing Unit (GPU). Simulations were carried out with various levels of fluctuation in initial conditions using the averaging method of events coming from Glauber Monte Carlo and UrQMD models. Energy density distributions were analyzed and the contribution of fluctuations in initial conditions was assessed in the hydrodynamic simulation.

        Speaker: Dr Marcin Slodkowski (Warsaw University of Technology (PL))
      • 16:15
        Testing the limits of HTTPS single point third party copy transfer over the WAN 15m

        LHC data is constantly being moved between computing and storage sites to support analysis, processing, and simulation; this is done at a scale that is currently unique within the science community. For example, the CMS experiment on the LHC manages approximately 200PB of data and, on a daily basis, moves 1PB between sites. This talk shows the performance results we have produced of exploring alternatives to the GridFTP protocol for these data movements. In particular the HTTPS third party copy over Xrootd data servers as a possible replacement of GridFTP for LHC big data movements.

        Speaker: Brian Paul Bockelman (University of Nebraska Lincoln (US))
      • 16:15
        TGenBase - general purpose database engine for HEP 15m

        TGenBase is a virtual database engine which allows to communicate with and store data in different underlying database management systems such as PostgreSQL, MySQL, SQLite, based on the configuration. It is universally applicable for any data storage task, such as parameter handling, detector component description, logistics, etc. In addition to usual CRUD (create, read, update, delete), it supports a special versioned insert-only logic, meaning that there is no need to update single records and the whole history of the records is available. The historical versions of the data can be queried with certain date as parameter.

        The core of the TGenBase is the data definition interface with template-based generation engine realized as a web-based application. It allows the end-user to define what and in which form they want the data to be stored and define relations between different data classes. Based on this definition, the database layout, server and client-side code is generated from templates and easily deployed. The server provides standard RESTful API, a middleware layer for user and access management, fast caching, etc. Data querying, visualization and modification are available in C++, Web, Python and LabVIEW "thin"-clients with minimum dependencies. The Web client provides full-fledged content management system with data access and administrative interface. For HEP, the C++ client is of high relevance, it can use ROOT framework to import and export data. Furthermore, STL containers and ROOT objects, such as complete histograms, can be stored in the DB along with primitive types.

        In this contribution we give an overview of the TGenBase functionality on an example of component quality control and logistics database for the Silicon Tracking System detector of the CBM experiment at FAIR.

        Speaker: Dr Evgeny Lavrik (Facility for Antiproton and Ion Research)
      • 16:15
        The Belle II Raw Data Management system 15m

        The Belle II experiment is a major upgrade of the e+e- asymmetric collider Belle, expected to produce tens of peta-bytes of data per year due to the luminosity increase with the SuperKEKB accelerator. The distributed computing system of the Belle II experiment plays a key role, storing and distributing data in a reliable way, to be easily access and analyzed along the more than 800 collaborators.
        In particular, the Belle II Raw Data Management system has been developed, aiming to upload the output files of the experiment onto Grid storage, register them into the file and metadata catalogs, and make two replicas of the full raw data set using the Belle II distributed data management system. It has been implemented as an extension of DIRAC (Distributed Infrastructure with Remote Agent Control), containing a database, services, client and monitoring tools, and several agents treating the data automatically.
        The first year of data taken with the Belle II full detector has been treated by the Belle II Raw Data Management system successfully. The design, current status and performance are presented. Prospects for improvements towards the full luminosity data taking are also reviewed.

        Speaker: Michel Hernandez Villanueva (University of Mississippi)
      • 16:15
        The updated DESGW processing pipeline for the third LIGO-VIRGO observing season 15m

        The DESGW group seeks to identify electromagnetic counterparts of gravitational wave events seen by the LIGO-VIRGO network, such as those expected from binary neutron star mergers or neutron star- black hole mergers. DESGW was active throughout the first two LIGO observing seasons, following up several binary black hole mergers and the first binary neutron star merger, GW170817. We describe the modifications to the observing strategy generation and image processing pipeline between the second (ending in August 2017) and third (beginning in April 2019) LIGO observing seasons. The modifications include a more robust observing strategy generator, further parallelization of the image reduction software and difference imaging processing pipeline, data transfer streamlining, and a public webpage listing identified counterpart candidates that updates in real time. Taken together, the additional parallelization steps enable us to identify potential electromagnetic counterparts within fully calibrated search images in less than one hour, compared to the 3-5 hours it would typically take during the first two seasons. These performance improvements are critical to the entire EM followup community, as rapid identification (or rejection) of candidates enables detailed spectroscopic followup by multiple instruments as soon as possible, leading to more information about the environment immediately following such gravitational wave events.

        Speaker: Kenneth Richard Herner (Fermi National Accelerator Laboratory (US))
      • 16:15
        The use of Convolutional Neural Networks for signal-background classification in Particle Physics experiments 15m

        The success of Convolutional Neural Networks (CNNs) in image classification has prompted efforts to study their use for classifying image data obtained in Particle Physics experiments.
        In this poster, I will discuss our efforts to apply CNNs to 3D image data from particle physics experiments to classify signal and background.

        In this work, we present an extensive 3D convolutional neural architecture search, achieving high accuracy for signal/background discrimination for a HEP classification use-case based on simulated data from the Ice Cube experiment detector and an ATLAS-like detector. We demonstrate among other things that we can achieve the same accuracy as complex ResNet architectures with 3D-CNNs with less parameters, and present comparisons of computational requirements, training and inference times.

        Speaker: Dr Venkitesh Ayyar (Lawrence Berkeley National Lab)
      • 16:15
        TMPIFile: A New Parallel I/O Solution in ROOT 15m

        Communication among processes is generating considerable interest in the scientific computing community due to the increasing use of distributed memory systems. In the field of high energy physics (HEP), however, little research has been addressed on this topic. More precisely in ROOT I/O, the de facto standard for data persistence in HEP applications, no such feature is provided. In order to perform efficient and robust cross-node communications, we introduce the TMPIFile functionality into ROOT where Message Passing Interface (MPI) is used to pass data across the entire distributed system. In the case of ATLAS workflows, instead of writing to file, the compressed data for each node is now stored into the sender side of TMPIFile. After a certain amount of data is collected, the TMPIFile sender can automatically package the data into a memory buffer and send it via MPI. On the other end of the communication, a collector receives buffers from multiple senders, merges them into one file, and at last, writes the file into the disk. Multiple collectors can be created to avoid I/O contention. Test results will be shown from runs at NERSC and ALCF.

        Speaker: Steven Farrell (Lawrence Berkeley National Lab (US))
      • 16:15
        Upgrade of CMS non-event data infrastructure for the Run 3 High Level Trigger 15m

        The CMS experiment at CERN is working to improve the selection capability of the High Level Trigger (HLT) system, in view of the re-start of the collisions for Run 3. One key factor on this scope is to enhance the ability of the Trigger to track the detector evolution during the data taking, along with the LHC Fill cycles. In particular, the HLT performance is sensitive to two areas of varying conditions: the evolution of the LHC luminous region parameters, changing frequently during the fill cycle, and the radiation-induced effects on the detectors, that become relevant within long data collection runs. In this context, the CMS Condition Database system has to fulfil the new requirements imposed, supporting the capability to deliver updates during the LHC fill with the required time granularity. In this contribution, we present the solution that CMS has implemented on the combined system HLT-Condition Database for these purposes. The main strategy, that substantially confirms the adoption of Oracle RDBMS technology ( for the data storage ) and Frontier Web Proxy Caches ( for the read access Tier ) will be described. The requirements of Latency, Consistency, and Reproducibility imposed by the HLT multi-process architecture will be discussed in detail. The results of the integration and stress tests will be also provided.

        Speaker: Remi Mommsen (Fermi National Accelerator Lab. (US))
      • 16:15
        Upgrade of the KEDR detector DAQ system 15m

        The KEDR experiment is ongoing at the VEPP-4M e+e- collider at Budker INP in Novosibirsk. The collider center of mass energy range covers wide area from 2 to 11 GeV. Most of the up-to-date statistics were taken at the lower end of the energy range around charmonia region.
        Planned activities at greater energies up to bottomonia would lead to significant rise of event recording rates and accelerator backgrounds, thus stressing the existing DAQ and trigger systems beyond their limits.

        To operate at higher energies, we need to:

        • reduce trigger decision time
        • reduce time of transferring digitized data from electronics
          to computers

        Described in the presentation DAQ upgrade plan includes:

        • Re-design of trigger electronics using modern components
          to improve trigger decision time
        • Development of new readout processors using ethernet connections
        • New software for collecting events and electronics management
        • High level of parallelization of data transfers and events processing
        • Improved reliability based on readout computing cluster with
          redundancy

        The upgraded DAQ system is very flexible and could be considered as a concept prototype for the perspective BINP project of Super Charm-Tau Factory.

        Speaker: Mr Dmitriy Maximov (Budker Institute of Nucler Physics)
      • 16:15
        Using the RichACL Standard for Access Control in EOS 15m

        The EOS storage system in use at CERN and several other HEP sites was developed with an access control system driven by known use cases, which is still in its infancy.

        Here we motivate the decision to strive supporting the RichACL standard as far as the EOS design allows. We highlight a characteristic that fits particularly well with access control for other applications at CERN, and show how the resemblance of RichACL to NFS V4 and Microsoft Windows ACLs led to a straightforward implementation of an access control module for Windows/Samba.

        Speaker: Andreas Joachim Peters (CERN)
      • 16:15
        Visualisations of the non-trivial QCD vacuum 15m

        Despite the success of quantum chromodynamics (QCD) in describing the strong nuclear force, a clear picture of how this theory gives rise to the distinctive properties of confinement and dynamical chiral symmetry breaking at low energy is yet to be found. One of the more promising models used to explain these phenomena in recent times is known as the centre vortex model. In this work we explore the properties of the gluon propagator in the context of this model, adding to the already substantial body of evidence supporting the importance of centre vortices in QCD. We also present novel visualisation techniques that have been devised to allow for detailed hands-on exploration of the centre-vortex structure of the QCD vacuum. These techniques provide new insight into the behaviour of centre vortices in low-energy lattice QCD.

        Speaker: Mr James Biddle (University of Adelaide)
      • 16:15
        Winventory: microservices architecture case study 15m

        In the CERN laboratory, users have access to a large number of different licensed software assets. The landscape of such assets is very heterogeneous including Windows operating systems, office tools and specialized technical and engineering software. In order to improve management of the licensed software and to understand better needs of the users, it was decided to develop a Winventory application. The Winventory is a tool that gathers and presents statistics of software assets on CERN Windows machines and facilitates interaction with their individual users. The system was built based on microservices architecture pattern, an increasingly popular approach to web application development. The microservices architecture pattern separates the application into multiple independently deployable units that can be individually developed, tested and deployed. This paper presents the microservices architecture and design choices made in order to achieve a modern, maintainable and extensible system for managing licensed software at CERN.

        Speaker: Sebastian Bukowiec (CERN)
    • 16:30 18:20
      Plenary: Molecular sciences / Analysis models / Diversity panel Hall G

      Hall G

      Adelaide Convention Centre

      Convener: Lucia Silvestris (Universita e INFN, Bari (IT))
      • 16:30
        Challenges and opportunities for the Computational Molecular Sciences 30m
        Speaker: Daniel Crawford (Virginia Tech/Molecular Sciences Software Institute)
      • 17:00
        The evolution of analysis models for HL-LHC 30m
        Speaker: Andrea Rizzi (INFN Sezione di Pisa, Universita' e Scuola Normale Superiore, P)
      • 17:30
        Diversity & inclusivity discussion panel 50m
        Speakers: Gordon Watts (University of Washington (US)), Maria Girone (CERN)
    • 18:20 18:50
      Table seating for banquet commences in Hall CD Lower at 18:30 30m Hall CD Lower

      Hall CD Lower

      Adelaide Convention Centre

    • 18:50 22:50
      Social Event: Banquet Hall CD Lower

      Hall CD Lower

      Adelaide Convention Centre

    • 08:55 10:30
      Plenary: Track highlights Hall G

      Hall G

      Adelaide Convention Centre

      Convener: Graeme A Stewart (CERN)
      • 08:55
        Introduction to the Track Highlights 5m
        Speaker: Graeme A Stewart (CERN)
      • 09:00
        Track 1 Highlights – Online and Real-time Computing 15m
        Speaker: Steven Schramm (Universite de Geneve (CH))
      • 09:15
        Track 2 Highlights – Offline Computing 15m
        Speaker: Paul James Laycock (Brookhaven National Laboratory (US))
      • 09:30
        Track 3 Highlights – Middleware and Distributed Computing 15m
        Speaker: Tomoe Kishimoto (University of Tokyo (JP))
      • 09:45
        Track 4 Highlights – Data Organisation, Management and Access 15m
        Speakers: Tigran Mkrtchyan (DESY), Tigran Mkrtchyan (Ruprecht Karls Universitaet Heidelberg (DE))
      • 10:00
        Track 5 Highlights – Software Development 15m
        Speaker: Martin Ritter (LMU / Cluster Universe)
      • 10:15
        Track 6 Highlights – Physics Analysis 15m
        Speaker: Phiala Shanahan (Massachusetts Institute of Technology)
    • 10:30 11:00
      Break 30m
    • 11:00 12:30
      Plenary: Track highlights / Closing Hall G

      Hall G

      Adelaide Convention Centre

      Convener: Waseem Kamleh (University of Adelaide)
      • 11:00
        Track 7 Highlights – Facilities, Clouds and Containers 15m
        Speaker: Christoph Wissing (Deutsches Elektronen-Synchrotron (DE))
      • 11:15
        Track 8 Highlights – Collaboration, Education, Training and Outreach 15m
        Speaker: Marzena Lapka (CERN)
      • 11:30
        Track 9 Highlights – Exascale Science 15m
        Speaker: Wei Yang (SLAC National Accelerator Laboratory (US))
      • 11:45
        Track X Highlights – Crossover sessions from online, offline and exascale 10m
        Speaker: Teng Jian Khoo (Universite de Geneve (CH))
      • 11:55
        CHEP 2021 20m
        Speaker: Amber Boehnlein (Jefferson Lab)
      • 12:15
        Closing 15m
        Speaker: Dr Waseem Kamleh (University of Adelaide)