Conveners
Track 5 – Software Development: General frameworks and schedulers
- Martin Ritter (LMU / Cluster Universe)
Track 5 – Software Development: Parallelism, accelerators and heterogenous computing
- Felice Pantaleo (CERN)
Track 5 – Software Development: Heterogenous computing and workflow managers
- Felice Pantaleo (CERN)
Track 5 – Software Development: Software management and packaging
- Mihaela Gheata (Institute of Space Science (RO))
Track 5 – Software Development: Testbeds and CI systems
- Jason Webb (Brookhaven National Lab)
Track 5 – Software Development: Common tools 1: GUI, geometry, analysis, data models
- Mihaela Gheata (Institute of Space Science (RO))
Track 5 – Software Development: Common tools 2: Messaging, declarative frameworks, optimizers
- Jason Webb (Brookhaven National Lab)
-
Gerardo Ganis (CERN)04/11/2019, 11:00Track 5 – Software DevelopmentOral
The Future Circular Collider (FCC) is designed to provide unprecedented luminosity and unprecedented centre-of-mass energies. The physics reach and potential of the different FCC options - $e^+e^-$, $pp$, $e^-p$ - has been studied and published in dedicated Conceptual Design Reports (CDRs) published at the end of 2018.
Go to contribution page
Conceptual detector designs have been developed for such studies and... -
Christopher Jones (Fermi National Accelerator Lab. (US))04/11/2019, 11:15Track 5 – Software DevelopmentOral
The diversity of the scientific goals across HEP experiments necessitates unique bodies of software tailored for achieving particular physics results. The challenge, however, is to identify the software that must be unique, and the code that is unnecessarily duplicated, which results in wasted effort and inhibits code maintainability.
Fermilab has a history of supporting and developing...
Go to contribution page -
Mohammad Al-Turany (GSI - Helmholtzzentrum fur Schwerionenforschung GmbH (DE))04/11/2019, 11:30Track 5 – Software DevelopmentOral
The ALFA framework is a joint development between ALICE Online-Offline and FairRoot teams. ALFA has a distributed architecture, i.e. a collection of highly maintainable, testable, loosely coupled, independently deployable processes.
ALFA allows the developer to focus on building single-function modules with well-defined interfaces and operations. The communication between the independent...
Go to contribution page -
Christopher Jones (Fermi National Accelerator Lab. (US))04/11/2019, 11:45Track 5 – Software DevelopmentOral
The OpenMP standard is the primary mechanism used at high performance computing facilities to allow intra-process parallelization. In contrast, many HEP specific software (such as CMSSW, GaudiHive, and ROOT) make use of Intel's Threading Building Blocks (TBB) library to accomplish the same goal. In this talk we will discuss our work to compare TBB and OpenMP when used for scheduling algorithms...
Go to contribution page -
Niklas Nolte (CERN / Technische Universitaet Dortmund (DE))04/11/2019, 12:00Track 5 – Software DevelopmentOral
The high-level trigger (HLT) of LHCb in Run 3 will have to process 5 TB/s of data, which is about two orders of magnitude larger compared to Run 2. The second stage of the HLT runs asynchronously to the LHC, aiming for a throughput of about 1 MHz. It selects analysis-ready physics signals by O(1000) dedicated selections totaling O(10000) algorithms to achieve maximum efficiency. This poses two...
Go to contribution page -
Daniel Hugo Campora Perez (Universidad de Sevilla (ES))04/11/2019, 12:15Track 5 – Software DevelopmentOral
As part of the LHCb detector upgrade in 2021, the hardware-level trigger will be removed, coinciding with an increase in luminosity. As a consequence, about 40 Tbit/s of data will be processed in a full-software trigger, a challenge that has prompted the exploration of alternative hardware technologies. Allen is a framework that permits concurrent many-event execution targeting many-core...
Go to contribution page -
Scott Snyder (Brookhaven National Laboratory (US))04/11/2019, 14:00Track 5 – Software DevelopmentOral
In preparation for Run 3 of the LHC, the ATLAS experiment is modifying its offline software to be fully multithreaded. An important part of this is data structures that can be efficiently and safely concurrently accessed from many threads. A standard way of achieving this is through mutual exclusion; however, the overhead from this can sometimes be excessive. Fully lockless implementations are...
Go to contribution page -
Remi Ete (DESY)04/11/2019, 14:15Track 5 – Software DevelopmentOral
Marlin is the event processing framework of the iLCSoft ecosystem. Originally developed
Go to contribution page
for the ILC more than 15 years ago, it is now widely used, e.g. by CLICdp, CEPC and
many test beam projects such as Calice, LCTPC and EU-Telescope. While Marlin is
lightweight and flexible it was originally designed for sequential processing only.
With MarlinMT we now evolved Marlin for parallel processing... -
Oliver Gutsche (Fermi National Accelerator Lab. (US))04/11/2019, 14:30Track 5 – Software DevelopmentOral
The advent of computing resources with co-processors, for example Graphics Processing Units (GPU) or Field-Programmable Gate Arrays (FPGA), for use cases like the CMS High-Level Trigger (HLT) or data processing at leadership-class supercomputers imposes challenges for the current data processing frameworks. These challenges include developing a model for algorithms to offload their...
Go to contribution page -
Andrea Bocci (CERN)04/11/2019, 14:45Track 5 – Software DevelopmentOral
As the mobile ecosystem has demonstrated, ARM processors and GPUs promise to deliver higher compute efficiency with a lower power consumption. One interesting platform to experiment with architectures different from a traditional x86 machine is the NVIDIA AGX Xavier SoC, that pairs a 64-bit ARM processor 8 cores with a Volta-class GPU with 512 CUDA cores. The CMS reconstruction software was...
Go to contribution page -
Attila Krasznahorkay (CERN)04/11/2019, 15:00Track 5 – Software DevelopmentOral
With GPUs and other kinds of accelerators becoming ever more accessible, High Performance Computing Centres all around the world using them ever more, ATLAS has to find the best way of making use of such accelerators in much of its computing.
Go to contribution page
Tests with GPUs -- mainly with CUDA -- have been performed in the past in the experiment. At that time the conclusion was that it was not advantageous... -
Mr Ziheng Chen (Northwestern University)04/11/2019, 15:15Track 5 – Software DevelopmentOral
The future upgraded High Luminosity LHC (HL-LHC) is expected to deliver about 5 times higher instantaneous luminosity than the present LHC, producing pile-up up to 200 interactions per bunch crossing. As a part of its phase-II upgrade program, the CMS collaboration is developing a new end-cap calorimeter system, the High Granularity Calorimeter (HGCAL), featuring highly-segmented hexagonal...
Go to contribution page -
Dr Maxim Gonchar (Joint Institute for Nuclear Research)05/11/2019, 11:00Track 5 – Software DevelopmentOral
GNA is a high performance fitter, designed to handle large scale models with big number of parameters. Following the data flow paradigm the model in GNA is built as directed acyclic graph. Each node (transformation) of the graph represents a function, that operates on vectorized data. A library of transformations, implementing various functions, is precompiled. The graph itself is assembled...
Go to contribution page -
Vardan Gyurjyan (Jefferson Lab)05/11/2019, 11:15Track 5 – Software DevelopmentOral
Modern hardware is trending towards increasingly parallel and heterogeneous architectures. Contemporary machine processors are spread across multiple sockets, where each socket can access some system memory faster than the rest, creating non-uniform memory access (NUMA). Efficiently utilizing these NUMA machines is becoming increasingly important. This paper examines latest Intel Skylake and...
Go to contribution page -
Vardan Gyurjyan (Jefferson Lab)05/11/2019, 11:30Track 5 – Software DevelopmentOral
The hardware landscape used in HEP and NP is changing from homogeneous multi-core systems towards heterogeneous systems with many different computing units, each with their own characteristics. To achieve data processing maximum performance the main challenge is to place the right computing on the right hardware.
Go to contribution page
In this paper we discuss CLAS12 charge particle tracking workload partitioning... -
Marcel Rieger (CERN)05/11/2019, 11:45Track 5 – Software DevelopmentOral
In particle physics, workflow management systems are primarily used as tailored solutions in dedicated areas such as Monte Carlo production. However, physicists performing data analyses are usually required to steer their individual workflows manually which is time-consuming and often leads to undocumented relations between particular workloads.
Go to contribution page
We present the luigi analysis workflow (law)... -
Miha Muskinja (Lawrence Berkeley National Lab. (US))05/11/2019, 12:00Track 5 – Software DevelopmentOral
The ATLAS experiment has successfully integrated High-Performance Computing (HPC) resources in its production system. Unlike the current generation of HPC systems, and the LHC computing grid, the next generation of supercomputers is expected to be extremely heterogeneous in nature: different systems will have radically different architectures, and most of them will provide partitions optimized...
Go to contribution page -
Dr Venkitesh Ayyar (Lawrence Berkeley National Lab)05/11/2019, 12:15Track 5 – Software DevelopmentOral
High Energy Physics experiments face unique challenges when running their computation on High Performance Computing (HPC) resources. The LZ dark matter detection experiment has two data centers, one each in the US and UK, to perform computations. Its US data center uses the HPC resources at NERSC.
Go to contribution page
In this talk, I will describe the current computational workflow of the LZ experiment, detailing... -
Graeme A Stewart (CERN)05/11/2019, 14:00Track 5 – Software DevelopmentOral
High-Energy Physics has evolved a rich set of software packages that need to work harmoniously to carry out the key software tasks needed by experiments. The problem of consistently building and deploying these software packages as a coherent software stack is one that is shared across the HEP community. To that end the HEP Software Foundation Packaging Working Group has worked to identify...
Go to contribution page -
Prof. Benda Xu (Tsinghua University)05/11/2019, 14:15Track 5 – Software DevelopmentOral
In big physics experiments, as simulation, reconstruction and analysis become more sophisticated, scientific reproducibility is not a trivial task. Software is one of the biggest challenges. Modularity is a common sense of software engineering to facilitate quality and reusability of code. However, that often introduces nested dependencies not obvious for physicists to work with. Package...
Go to contribution page -
Elizabeth Sexton-Kennedy (Fermi National Accelerator Lab. (US))05/11/2019, 14:30Track 5 – Software DevelopmentOral
Development of scientific software has always presented challenges to its practitioners, among other things due to its inherently collaborative nature. Software systems often consistent of up to several dozen closely-related packages developed within a particular experiment or related ecosystem, with up to a couple of hundred externally-sourced dependencies. Making improvements to one such...
Go to contribution page -
Chris Burr (CERN)05/11/2019, 14:45Track 5 – Software DevelopmentOral
The conda package manager is widely used in both commercial and academic high-performance computing across a wide range of fields. In 2016 conda-forge was founded as a community-driven package repository which allows packaging efforts to be shared across communities. This is especially important with the challenges faced when packaging modern software with complex dependency chains or...
Go to contribution page -
Tamas Bato (CERN)05/11/2019, 15:00Track 5 – Software DevelopmentOral
The number of BYOD continuously grows at CERN. Additionally, it is desirable to move from a centrally managed model to a distributed model where users are responsible for their own devices. Following this strategy, the new tools have to be provided to distribute and - in case of licensed software - also track applications used by CERN users. The available open source and commercial solutions...
Go to contribution page -
Sebastian Bukowiec (CERN)05/11/2019, 15:15Track 5 – Software DevelopmentOral
CERN Windows server infrastructure consists of about 900 servers. The management and maintenance is often a challenging task as the data to be monitored is disparate and has to be collected from various sources. Currently, alarms are collected from the Microsoft System Center Operation Manager (SCOM) and many administrative actions are triggered through e-mails sent by various systems or...
Go to contribution page -
Robert William Gardner Jr (University of Chicago (US))05/11/2019, 16:30Track 5 – Software DevelopmentOral
The Scalable Systems Laboratory (SSL), part of the IRIS-HEP Software Institute, provides Institute participants and HEP software developers generally with a means to transition their R&D from conceptual toys to testbeds to production-scale prototypes. The SSL enables tooling, infrastructure, and services supporting innovation of novel analysis and data architectures, development of software...
Go to contribution page -
Rosen Matev (CERN)05/11/2019, 16:45Track 5 – Software DevelopmentOral
High energy physics experiments traditionally have large software codebases primarily written in C++ and the LHCb physics software stack is no exception. Compiling from scratch can easily take 5 hours or more for the full stack even on an 8-core VM. In a development workflow, incremental builds are often not sufficient for quick compilation on a typical PC (e.g. due to changes to headers or...
Go to contribution page -
Barbara Martelli (INFN CNAF)05/11/2019, 17:00Track 5 – Software DevelopmentOral
Software defect prediction aims at detecting part of software that can likely contain faulty modules - e.g. in terms of complexity, maintainability, and other software characteristics - and therefore that require actual attention. Machine Learning (ML) has proven to be of great value in a variety of Software Engineering tasks, such as software defects prediction, also in the presence of...
Go to contribution page -
Walter Lampl (University of Arizona (US))05/11/2019, 17:15Track 5 – Software DevelopmentOral
The ART system is designed to run test jobs on the Grid after an ATLAS nightly release has been built. The choice was taken to exploit the Grid as a backend as it offers a huge resource pool, suitable for a deep set of integration tests, and running the tests could be delegated to the highly scalable ATLAS production system (PanDA). The challenge of enabling the Grid as a test environment is...
Go to contribution page -
Dr Robert Andrew Currie (The University of Edinburgh (GB))05/11/2019, 17:30Track 5 – Software DevelopmentOral
The physics software stack of LHCb is based on Gaudi and is comprised of about 20 interdependent projects, managed across multiple Gitlab repositories. At present, the continuous integration (CI) system used for regular building and testing of this software is implemented using Jenkins and runs on a cluster of about 300 cores.
LHCb CI pipelines are python-based and relatively modern with some...
Go to contribution page -
Mircho Nikolaev Rodozov (Bulgarian Academy of Sciences (BG))05/11/2019, 17:45Track 5 – Software DevelopmentOral
The CMS experiment relies on a substantial C++ and Python-based software release for its day-to-day production, operations and analysis needs. While very much under active development, this codebase continues to age. At the same time, CMSSW codes are likely to be used for the next two decades, in one form or another. Thus, the "cost" of bugs entering CMSSW continues to increase, both due to...
Go to contribution page -
Serguei Linev (GSI - Helmholtzzentrum fur Schwerionenforschung GmbH (DE))07/11/2019, 11:00Track 5 – Software DevelopmentOral
RWebWindow class builds the core functionality for web-based widgets in ROOT. It combines all necessary server-side components and provides communication channels with multiple JavaScript clients.
Following new ROOT widgets are build based on RWebWindow functionality:
- RCanvas – ROOT7 canvas for drawing all kinds of primitives, including
histograms and graphs - RBrowser – hierarchical...
- RCanvas – ROOT7 canvas for drawing all kinds of primitives, including
-
Henry Fredrick Schreiner (University of Cincinnati (US))07/11/2019, 11:15Track 5 – Software DevelopmentOral
Boost.Histogram, a header-only C++14 library that provides multi-dimensional histograms and profiles, is now available in Boost-1.70. It is extensible, fast, and uses modern C++ features. Using template meta-programming, the most efficient code path for any given configuration is automatically selected. The library includes key features designed for the particle physics community, such as...
Go to contribution page -
Oksana Shadura (University of Nebraska Lincoln (US))07/11/2019, 11:30Track 5 – Software DevelopmentOral
C++ Modules come in C++20 to fix the long-standing build scalability problems in the language. They provide an io-efficient, on-disk representation capable to reduce build times and peak memory usage. ROOT employs the C++ modules technology further in the ROOT dictionary system to improve its performance and reduce the memory footprint.
ROOT with C++ Modules was released as a technology...
Go to contribution page -
Frank-Dieter Gaede (Deutsches Elektronen-Synchrotron (DE))07/11/2019, 11:45Track 5 – Software DevelopmentOral
PODIO is a C++ toolkit for the creation of event data models (EDMs) with a fast and efficient I/O layer, developed in the AIDA2020 project. It employs plain-old-data (POD) data structures wherever possible, while avoiding deep object-hierarchies and virtual inheritance. A lightweight layer of handle classes provides the necessary high-level interface for the physicist, such as support for...
Go to contribution page -
Serguei Linev (GSI - Helmholtzzentrum fur Schwerionenforschung GmbH (DE))07/11/2019, 12:00Track 5 – Software DevelopmentOral
REve library of ROOT provides web-based event display and includes all necessary components for geometry visualization. These components are reused in the new web-based geometry viewer, where ROOT geometries of arbitrary complexity can be displayed.
Go to contribution page
With new geometry viewer one could browse hierarchy of the geometry nodes, change individual node/volume/shape attributes, search volumes by name... -
Norman Anthony Graf (SLAC National Accelerator Laboratory (US))07/11/2019, 12:15Track 5 – Software DevelopmentOral
Geant4 is the de facto HEP standard for simulating the interaction of particles with materials and fields. The software toolkit provides a very rich library of basic geometrical shapes, often referred to as “primitives”, plus the ability to define compound geometries, making it capable of supporting extremely complex physical structures. The ability to directly import CAD geometries into...
Go to contribution page -
Jorn Schumacher (CERN)07/11/2019, 14:00Track 5 – Software DevelopmentOral
NetIO is a network communication library that enables distributed applications to exchange messages using high-level communication patterns such as publish/subscribe. NetIO is based on libfabric and supports various types of RDMA networks, for example, Infiniband, RoCE, or OmniPath. NetIO is currently being used in the data acquisition chain of the ATLAS experiment.
Major parts of NetIO...
Go to contribution page -
Mr Tigran Mkrtchyan (DESY)07/11/2019, 14:15Track 5 – Software DevelopmentOral
As a well established, large-scale distributed storage system, dCache is required to manage and serve huge amount of data for WLHC experiments and beyond. Based on a microservices-like architecture, dCache is built as a modular system distributed, where each component provides a different core functionality. These services communicate by passing serialized messages of dynamic types to each...
Go to contribution page -
Andrew Bohdan Hanushevsky (SLAC National Accelerator Laboratory (US))07/11/2019, 14:30Track 5 – Software DevelopmentOral
One of the key components of the XRootD software framework is the C++ implementation of the XRootD client. As the foundation of client binaries, XRootD Posix API and the Python API, it is widely used in LHC experiments’ frameworks as well as on server side in the XCache and EOS. In order to facilitate new developments the XRootD client API has been extended to be in line with modern C++...
Go to contribution page -
Joshua Heneage Dawes (University of Manchester (GB))07/11/2019, 14:45Track 5 – Software DevelopmentOral
VyPR (http://cern.ch/vypr) is a framework being developed with the aim of automating as much as possible the performance analysis of Python programs. To achieve this, it uses an analysis-by-specification approach; developers specify the performance requirements of their programs (without any modifications of the source code) and such requirements are checked at runtime. VyPR then provides...
Go to contribution page -
Jim Pivarski (Princeton University)07/11/2019, 15:00Track 5 – Software DevelopmentOral
Over the past two years, the uproot library has become widely adopted among particle physicists doing analysis in Python. Rather than presenting an event model, uproot gives the user an array for each particle attribute. In case of multiple particles per event, this array is jagged: an array of unequal-length subarrays. Data structures and operations for manipulating jagged arrays are provided...
Go to contribution page -
Caterina Marcon (Lund University (SE))07/11/2019, 15:15Track 5 – Software DevelopmentOral
Experimental observations and advanced computer simulations in High Energy Physics (HEP) paved way for the recent discoveries at the Large Hadron Collider (LHC) at CERN. Currently, Monte Carlo simulations account for a very significant amount of computational resources of the Worldwide LHC Computing Grid (WLCG).
Go to contribution page
In looking at the recent trends in modern computer architectures we see a...