Conveners
Track 5 – Software Development: General frameworks and schedulers
- Martin Ritter (LMU / Cluster Universe)
Track 5 – Software Development: Parallelism, accelerators and heterogenous computing
- Felice Pantaleo (CERN)
Track 5 – Software Development: Heterogenous computing and workflow managers
- Felice Pantaleo (CERN)
Track 5 – Software Development: Software management and packaging
- Mihaela Gheata (Institute of Space Science (RO))
Track 5 – Software Development: Testbeds and CI systems
- Jason Webb (Brookhaven National Lab)
Track 5 – Software Development: Common tools 1: GUI, geometry, analysis, data models
- Mihaela Gheata (Institute of Space Science (RO))
Track 5 – Software Development: Common tools 2: Messaging, declarative frameworks, optimizers
- Jason Webb (Brookhaven National Lab)
The Future Circular Collider (FCC) is designed to provide unprecedented luminosity and unprecedented centre-of-mass energies. The physics reach and potential of the different FCC options - $e^+e^-$, $pp$, $e^-p$ - has been studied and published in dedicated Conceptual Design Reports (CDRs) published at the end of 2018.
Conceptual detector designs have been developed for such studies and...
The diversity of the scientific goals across HEP experiments necessitates unique bodies of software tailored for achieving particular physics results. The challenge, however, is to identify the software that must be unique, and the code that is unnecessarily duplicated, which results in wasted effort and inhibits code maintainability.
Fermilab has a history of supporting and developing...
The ALFA framework is a joint development between ALICE Online-Offline and FairRoot teams. ALFA has a distributed architecture, i.e. a collection of highly maintainable, testable, loosely coupled, independently deployable processes.
ALFA allows the developer to focus on building single-function modules with well-defined interfaces and operations. The communication between the independent...
The OpenMP standard is the primary mechanism used at high performance computing facilities to allow intra-process parallelization. In contrast, many HEP specific software (such as CMSSW, GaudiHive, and ROOT) make use of Intel's Threading Building Blocks (TBB) library to accomplish the same goal. In this talk we will discuss our work to compare TBB and OpenMP when used for scheduling algorithms...
The high-level trigger (HLT) of LHCb in Run 3 will have to process 5 TB/s of data, which is about two orders of magnitude larger compared to Run 2. The second stage of the HLT runs asynchronously to the LHC, aiming for a throughput of about 1 MHz. It selects analysis-ready physics signals by O(1000) dedicated selections totaling O(10000) algorithms to achieve maximum efficiency. This poses two...
As part of the LHCb detector upgrade in 2021, the hardware-level trigger will be removed, coinciding with an increase in luminosity. As a consequence, about 40 Tbit/s of data will be processed in a full-software trigger, a challenge that has prompted the exploration of alternative hardware technologies. Allen is a framework that permits concurrent many-event execution targeting many-core...
In preparation for Run 3 of the LHC, the ATLAS experiment is modifying its offline software to be fully multithreaded. An important part of this is data structures that can be efficiently and safely concurrently accessed from many threads. A standard way of achieving this is through mutual exclusion; however, the overhead from this can sometimes be excessive. Fully lockless implementations are...
Marlin is the event processing framework of the iLCSoft ecosystem. Originally developed
for the ILC more than 15 years ago, it is now widely used, e.g. by CLICdp, CEPC and
many test beam projects such as Calice, LCTPC and EU-Telescope. While Marlin is
lightweight and flexible it was originally designed for sequential processing only.
With MarlinMT we now evolved Marlin for parallel processing...
The advent of computing resources with co-processors, for example Graphics Processing Units (GPU) or Field-Programmable Gate Arrays (FPGA), for use cases like the CMS High-Level Trigger (HLT) or data processing at leadership-class supercomputers imposes challenges for the current data processing frameworks. These challenges include developing a model for algorithms to offload their...
As the mobile ecosystem has demonstrated, ARM processors and GPUs promise to deliver higher compute efficiency with a lower power consumption. One interesting platform to experiment with architectures different from a traditional x86 machine is the NVIDIA AGX Xavier SoC, that pairs a 64-bit ARM processor 8 cores with a Volta-class GPU with 512 CUDA cores. The CMS reconstruction software was...
With GPUs and other kinds of accelerators becoming ever more accessible, High Performance Computing Centres all around the world using them ever more, ATLAS has to find the best way of making use of such accelerators in much of its computing.
Tests with GPUs -- mainly with CUDA -- have been performed in the past in the experiment. At that time the conclusion was that it was not advantageous...
The future upgraded High Luminosity LHC (HL-LHC) is expected to deliver about 5 times higher instantaneous luminosity than the present LHC, producing pile-up up to 200 interactions per bunch crossing. As a part of its phase-II upgrade program, the CMS collaboration is developing a new end-cap calorimeter system, the High Granularity Calorimeter (HGCAL), featuring highly-segmented hexagonal...
GNA is a high performance fitter, designed to handle large scale models with big number of parameters. Following the data flow paradigm the model in GNA is built as directed acyclic graph. Each node (transformation) of the graph represents a function, that operates on vectorized data. A library of transformations, implementing various functions, is precompiled. The graph itself is assembled...
Modern hardware is trending towards increasingly parallel and heterogeneous architectures. Contemporary machine processors are spread across multiple sockets, where each socket can access some system memory faster than the rest, creating non-uniform memory access (NUMA). Efficiently utilizing these NUMA machines is becoming increasingly important. This paper examines latest Intel Skylake and...
The hardware landscape used in HEP and NP is changing from homogeneous multi-core systems towards heterogeneous systems with many different computing units, each with their own characteristics. To achieve data processing maximum performance the main challenge is to place the right computing on the right hardware.
In this paper we discuss CLAS12 charge particle tracking workload partitioning...
In particle physics, workflow management systems are primarily used as tailored solutions in dedicated areas such as Monte Carlo production. However, physicists performing data analyses are usually required to steer their individual workflows manually which is time-consuming and often leads to undocumented relations between particular workloads.
We present the luigi analysis workflow (law)...
The ATLAS experiment has successfully integrated High-Performance Computing (HPC) resources in its production system. Unlike the current generation of HPC systems, and the LHC computing grid, the next generation of supercomputers is expected to be extremely heterogeneous in nature: different systems will have radically different architectures, and most of them will provide partitions optimized...
High Energy Physics experiments face unique challenges when running their computation on High Performance Computing (HPC) resources. The LZ dark matter detection experiment has two data centers, one each in the US and UK, to perform computations. Its US data center uses the HPC resources at NERSC.
In this talk, I will describe the current computational workflow of the LZ experiment, detailing...
High-Energy Physics has evolved a rich set of software packages that need to work harmoniously to carry out the key software tasks needed by experiments. The problem of consistently building and deploying these software packages as a coherent software stack is one that is shared across the HEP community. To that end the HEP Software Foundation Packaging Working Group has worked to identify...
In big physics experiments, as simulation, reconstruction and analysis become more sophisticated, scientific reproducibility is not a trivial task. Software is one of the biggest challenges. Modularity is a common sense of software engineering to facilitate quality and reusability of code. However, that often introduces nested dependencies not obvious for physicists to work with. Package...
Development of scientific software has always presented challenges to its practitioners, among other things due to its inherently collaborative nature. Software systems often consistent of up to several dozen closely-related packages developed within a particular experiment or related ecosystem, with up to a couple of hundred externally-sourced dependencies. Making improvements to one such...
The conda package manager is widely used in both commercial and academic high-performance computing across a wide range of fields. In 2016 conda-forge was founded as a community-driven package repository which allows packaging efforts to be shared across communities. This is especially important with the challenges faced when packaging modern software with complex dependency chains or...
The number of BYOD continuously grows at CERN. Additionally, it is desirable to move from a centrally managed model to a distributed model where users are responsible for their own devices. Following this strategy, the new tools have to be provided to distribute and - in case of licensed software - also track applications used by CERN users. The available open source and commercial solutions...
CERN Windows server infrastructure consists of about 900 servers. The management and maintenance is often a challenging task as the data to be monitored is disparate and has to be collected from various sources. Currently, alarms are collected from the Microsoft System Center Operation Manager (SCOM) and many administrative actions are triggered through e-mails sent by various systems or...
The Scalable Systems Laboratory (SSL), part of the IRIS-HEP Software Institute, provides Institute participants and HEP software developers generally with a means to transition their R&D from conceptual toys to testbeds to production-scale prototypes. The SSL enables tooling, infrastructure, and services supporting innovation of novel analysis and data architectures, development of software...
High energy physics experiments traditionally have large software codebases primarily written in C++ and the LHCb physics software stack is no exception. Compiling from scratch can easily take 5 hours or more for the full stack even on an 8-core VM. In a development workflow, incremental builds are often not sufficient for quick compilation on a typical PC (e.g. due to changes to headers or...
Software defect prediction aims at detecting part of software that can likely contain faulty modules - e.g. in terms of complexity, maintainability, and other software characteristics - and therefore that require actual attention. Machine Learning (ML) has proven to be of great value in a variety of Software Engineering tasks, such as software defects prediction, also in the presence of...
The ART system is designed to run test jobs on the Grid after an ATLAS nightly release has been built. The choice was taken to exploit the Grid as a backend as it offers a huge resource pool, suitable for a deep set of integration tests, and running the tests could be delegated to the highly scalable ATLAS production system (PanDA). The challenge of enabling the Grid as a test environment is...
The physics software stack of LHCb is based on Gaudi and is comprised of about 20 interdependent projects, managed across multiple Gitlab repositories. At present, the continuous integration (CI) system used for regular building and testing of this software is implemented using Jenkins and runs on a cluster of about 300 cores.
LHCb CI pipelines are python-based and relatively modern with some...
The CMS experiment relies on a substantial C++ and Python-based software release for its day-to-day production, operations and analysis needs. While very much under active development, this codebase continues to age. At the same time, CMSSW codes are likely to be used for the next two decades, in one form or another. Thus, the "cost" of bugs entering CMSSW continues to increase, both due to...
RWebWindow class builds the core functionality for web-based widgets in ROOT. It combines all necessary server-side components and provides communication channels with multiple JavaScript clients.
Following new ROOT widgets are build based on RWebWindow functionality:
- RCanvas – ROOT7 canvas for drawing all kinds of primitives, including
histograms and graphs - RBrowser – hierarchical...
Boost.Histogram, a header-only C++14 library that provides multi-dimensional histograms and profiles, is now available in Boost-1.70. It is extensible, fast, and uses modern C++ features. Using template meta-programming, the most efficient code path for any given configuration is automatically selected. The library includes key features designed for the particle physics community, such as...
C++ Modules come in C++20 to fix the long-standing build scalability problems in the language. They provide an io-efficient, on-disk representation capable to reduce build times and peak memory usage. ROOT employs the C++ modules technology further in the ROOT dictionary system to improve its performance and reduce the memory footprint.
ROOT with C++ Modules was released as a technology...
PODIO is a C++ toolkit for the creation of event data models (EDMs) with a fast and efficient I/O layer, developed in the AIDA2020 project. It employs plain-old-data (POD) data structures wherever possible, while avoiding deep object-hierarchies and virtual inheritance. A lightweight layer of handle classes provides the necessary high-level interface for the physicist, such as support for...
REve library of ROOT provides web-based event display and includes all necessary components for geometry visualization. These components are reused in the new web-based geometry viewer, where ROOT geometries of arbitrary complexity can be displayed.
With new geometry viewer one could browse hierarchy of the geometry nodes, change individual node/volume/shape attributes, search volumes by name...
Geant4 is the de facto HEP standard for simulating the interaction of particles with materials and fields. The software toolkit provides a very rich library of basic geometrical shapes, often referred to as “primitives”, plus the ability to define compound geometries, making it capable of supporting extremely complex physical structures. The ability to directly import CAD geometries into...
NetIO is a network communication library that enables distributed applications to exchange messages using high-level communication patterns such as publish/subscribe. NetIO is based on libfabric and supports various types of RDMA networks, for example, Infiniband, RoCE, or OmniPath. NetIO is currently being used in the data acquisition chain of the ATLAS experiment.
Major parts of NetIO...
As a well established, large-scale distributed storage system, dCache is required to manage and serve huge amount of data for WLHC experiments and beyond. Based on a microservices-like architecture, dCache is built as a modular system distributed, where each component provides a different core functionality. These services communicate by passing serialized messages of dynamic types to each...
One of the key components of the XRootD software framework is the C++ implementation of the XRootD client. As the foundation of client binaries, XRootD Posix API and the Python API, it is widely used in LHC experiments’ frameworks as well as on server side in the XCache and EOS. In order to facilitate new developments the XRootD client API has been extended to be in line with modern C++...
VyPR (http://cern.ch/vypr) is a framework being developed with the aim of automating as much as possible the performance analysis of Python programs. To achieve this, it uses an analysis-by-specification approach; developers specify the performance requirements of their programs (without any modifications of the source code) and such requirements are checked at runtime. VyPR then provides...
Over the past two years, the uproot library has become widely adopted among particle physicists doing analysis in Python. Rather than presenting an event model, uproot gives the user an array for each particle attribute. In case of multiple particles per event, this array is jagged: an array of unequal-length subarrays. Data structures and operations for manipulating jagged arrays are provided...
Experimental observations and advanced computer simulations in High Energy Physics (HEP) paved way for the recent discoveries at the Large Hadron Collider (LHC) at CERN. Currently, Monte Carlo simulations account for a very significant amount of computational resources of the Worldwide LHC Computing Grid (WLCG).
In looking at the recent trends in modern computer architectures we see a...
