Nov 4 – 8, 2019
Adelaide Convention Centre
Australia/Adelaide timezone

SpackDev: Parallel Package Development with Spack

Nov 5, 2019, 2:30 PM
15m
Riverbank R2 (Adelaide Convention Centre)

Riverbank R2

Adelaide Convention Centre

Oral Track 5 – Software Development Track 5 – Software Development

Speaker

Elizabeth Sexton-Kennedy (Fermi National Accelerator Lab. (US))

Description

Development of scientific software has always presented challenges to its practitioners, among other things due to its inherently collaborative nature. Software systems often consistent of up to several dozen closely-related packages developed within a particular experiment or related ecosystem, with up to a couple of hundred externally-sourced dependencies. Making improvements to one such package can require related changes to multiple other packages, and some systemic improvements can require major structural changes across the ecosystem.

There have been several attempts to produce a multi-package development system within HEP in the past, such systems usually being limited to one or a few experiments and requiring a common build system (e.g. Make, CMake). Common features include a central installation of each "release" of the software system to avoid multiple builds of the same package on a system, and integration with version control systems.

SpackDev is based on the powerful Spack build and packaging system in wide use in HPC, utilizing its package recipes and build mangement system to extract build instructions and manage the parallel development, build and test process for multiple packages at a time. Intended to handle packages without restriction to one internal build system, SpackDev is integrated with Spack as a command extension and is generally applicable outside HEP. We describe SpackDev's features and development over the last two years and the medium-term future, and initial experience using the SpackDev in the context of the LArSoft liquid argon detector toolkit.

Consider for promotion No

Primary authors

Dr Christopher Green (Fermi National Accelerator Laboratory) Dr James Amundson (Fermilab)

Co-authors

Lynn Garren (Fermi National Accelerator Lab. (US)) Dr Patrick Gartung (Fermilab (US)) Ken Herner (SUNY Stony Brook) Elizabeth Sexton-Kennedy (Fermi National Accelerator Lab. (US))

Presentation materials