ASTERICS-OBELICS PyGamma19 - Python and open data for gamma-ray astronomy

18 Mar 2019, 12:00 → 22 Mar 2019, 15:30 Europe/Berlin

MPI for Nuclear Physics

 

 

 

 

 

ASTERICS-OBELICS PyGamma19 - Python and open data for gamma-ray astronomy

Python-based tools are increasingly gaining acceptance and importance in astronomy. At the same time, the clear trend is towards interoperable and open data. The goal of this meeting is to discuss the status and prospects for gamma-ray astronomy and related fields.

The meeting takes place from Monday, March 18, 2019 noon to Friday, March 22 noon at the MPI for Nuclear Physics in Heidelberg, Germany.

"Registration" is open, see the "Travel and Accommodation" page.
"Call for abstracts" is also still open, we hope to finalise the program soon.

A preliminary Contribution list and Timetable is available here on Indico.

If you have any questions, please contact us any time.

---

We invite presentations that describe the design, scope, status, plans and challenges of open-source Python codes in gamma-ray astronomy such as e.g. Gammapy, ctools, ctapipe or km3pipe, as well as more general Python packages that are also used in gamma-ray astronomy such as e.g. Astropy, Sherpa, 3ML and Naima.

Other topics were collaboration and interoperability can be fostered are well-defined, possible even shared, high-level data models and formats. The gamma-astro-data-formats effort was started at PyGamma15 and has seen some limited adoption in prototypes in the IACT community since, but no "standard" has emerged so far. We'd like to invite presentations on the data model and formats in use from existing and planned gamma-ray telescopes such as e.g. CTA, Fermi, H.E.S.S., VERITAS, MAGIC, HAWC or INTEGRAL.

We also welcome presentations that give an overview and share experiences concerning high-level science tools and data from adjacent astroparticle and astrophysics fields. It would also be good to have a few deep-dive technical presentations on scientific computing with Python and Numpy, or newer technologies like Numba, Dask, Cupy, Pytorch or Tensorflow that show great potential or already are used for astronomical data analysis. Two specific topics we'd like to cover, because there is a direct need and potential for collaboration on basic functionality needed by several projects, are HEALPix and the in-development astropy-healpix, as well as a discussion on status, issues and possible improvements concerning optimisation, modeling & fitting in Python.

Please note that this is not a school, no hands-on tutorials are planned. We hope that every participant, from the student to the expert level, will gain a deeper understanding of what exists, and that the discussions and connections at PyGamma19 will give a boost to Python and open data in gamma-ray astronomy.

---

• Program organisation: Christoph Deil, Catherine Boisson, Florian Gaté, Jeremy Perkins, Kay Graf, Karl Kosack, Petra Hüntemeyer, Thomas Robitaille
• Local organisation: Ruth Crespo, Roberta Zanin, Axel Donath, Jason Watson, Christoph Deil, Jayesh Wagh

---

Who is organizing this workshop:  This workshop is co-organized by MPI for Nuclear Physics in Heidelberg, Germany and CNRS-LAPP. This workshop is organized in the framework of OBELICS  (Observatory E-environments LINked by common ChallengeS) work package of H2020-ASTERICS project.

 

 

 

 

 

 

H2020-ASTERICS: the Astronomy ESFRI and Research Infrastructure Cluster, ASTERICS, is a Research Infrastructure funded by the European Commission's Horizon 2020 framework. ASTERICS aims to address the cross-cutting synergies and common challenges shared by world class Astronomy ESFRI Facilities such as SKA, CTA, KM3NeT, LSST, LOFAR, EUCLID, EGO & ELT.               

For more information visit https://www.asterics2020.eu/

 

OBELICS: OBELICS stands for OBservatory E-environments LInked with common ChallengeS. OBELICS is one of the workpackages of H2020-ASTERICS. It aims at enabling interoperability and software re-use for the data generation, integration and analysis of the cluster of ESFRI and pathfinder facilities in Astronomy and Astroparticle Physics.

Monday, 18 March

12:00 Arrive

Participants arrive, get a certificate and badge, sort out WIFI.
If you want, go to lunch with MPIK people to EMBL canteen (at your own cost).

Monday: Afternoon 1

1 Welcome

Speaker: Prof. Jim Hinton (Max Planck Institute for Nuclear Physics)

2 Introduction

Context, goals, overview, logistics for PyGamma19

Speaker: Roberta Zanin (MPI for Nuclear Physics, Heidelberg)

20190318_PyGamma2019_Intro_RZ.pdf

3 Fermi-LAT

Speaker: Jeremy Perkins (NASA/GSFC)

Google Slides

Large Area Telescope pyGamma19.pdf

4 Cherenkov Telescope Array

Speaker: Matthias Füssling (CTAO)

2019_03_pygamma_CTA.pdf

16:00 Coffee

Monday: Afternoon 2

5 The Astropy Project

The Astropy Project is a community effort to develop a single core package for Astronomy in Python and foster interoperability between Python astronomy packages. In this talk I will give an introduction to the project, including an overview of progress so far, and I will outline our current plans for the future. I will also describe the Astropy coordinated and affiliated package ecosystem, which is formed by Python packages that are not part of the core package but integrate with Astropy, and are maintained by the Astropy team and the wider community.

Speaker: Thomas Robitaille (Aperio Software Ltd.)

Astropy.key

Astropy.pdf

6 Python in the CASA project and its pipelines

CASA, the Common Astronomy Software Applications, is a package for radio astronomical telescopes that is developed by an international consortium of institutions from America, Asia, Australia and Europe. It is used to manipulate and process both interferometric and single dish data, and is the primary data processing software for the Karl G. Jansky Very Large Array (VLA) as well as the Atacama Large Millimeter/submillimeter Array (ALMA). CASA consists of a suite of libraries and tools written in C++ which provide functionality exposed to Python. I will give an overview of the project, its origins, present and future plans. Topics will include telescopes and data formats supported, and how the software is built and distributed in its current 5.x series and the plans for the upcoming 6.x series. We will also see how the pipelines for automated data processing used by the VLA and ALMA have been developed entirely in Python since their inception.

Speaker: Federico Montesino Pouzols (European Southern Observatory)

casa_pipelines_slides_pygamma19.pdf

18:00 Welcome Reception

Food & drinks at MPIK

Tuesday, 19 March

09:00 Coffee

Tuesday: Morning 1

7 CDS HEALPix libraries

After a brief presentation of HEALPix, I will discuss the specificities of the CDS implementation.
This implementation has been natively coded in both Java and Rust. Wrappers make the Rust library accessible in Javascript/WebAssembly and Python.
Particular emphasis will be placed on the exact solution of cone/polygon coverage queries and on MOCs in which each cell contains a boolean flag.

Speaker: francois-xavier pineau (Université de Strasbourg)

healpix_fxpineau.pdf

8 Rust and its usage as Python extensions

I will give a brief presentation about the new Rust system programming langage:
- the compiler rules enabling memory safety
- how one can interface Rust code with Python

Then, regarding to what has been currently done in cdshealpix I will talk about the CI pipeline I used for generating and deploying binary wheels for different platforms.

Speaker: Mr Matthieu Baumann (CDS (Centre de Données de Strasbourg))

slides.pdf

9 Multi Order Coverage maps and Python

I will present MOC (Multi Order Coverage maps), a Virtual Observatory standard allowing one to describe and compare spatial coverage of astronomical datasets.

After presenting the MOC format (based on HEALPix tessellation), I will show some practical applications of MOC usage, using the MOCPy library developed at CDS.

Speaker: Mr Thomas Boch (CDS - Observatoire de Strasbourg)

MOC-Python-PyGamma19-TBoch.pdf

11:00 Coffee

Tuesday: Morning 2

10 Towards open and reproducible multi-instrument analysis in gamma-ray astronomy

Analysis and combination of data from different gamma-ray instruments involves the use of collaboration proprietary software and case-by-case methods.
By defining a common open format for high-level gamma-ray data (containing event lists and instrument response functions, using the FITS standard) we allow multi-instrument analysis within the context of open-source software.
This project aims to perform the first fully-reproducible, multi-instrument very-high-energy gamma-ray analysis.
Data of Fermi-LAT, MAGIC, VERITAS, H.E.S.S., and FACT, compliant with a preliminary common format (DL3) and analysed with the gammapy science tools, were used to produce a first joint fit of the Crab Nebula spectrum. Aspects of the error evaluation and the release format of a spectral measurement are also included in the discussion.
The talk will illustrate how to realize a fully reproducible publication relying on open access assets as GitHub repositories (https://github.com/open-gamma-ray-astro/joint-crab) and will promote and review current open data format and science tools to allow VHE gamma-ray astronomy to move towards open reproducible science.

Speaker: Cosimo Nigro

joint_crab_PyGamma19.pdf

11 Maps in gamma-ray astronomy

In several branches of astrophysics, high-level data format consists of event lists - containing the reconstructed energy, direction and arrival time (and some other informations, eg: event type) of each event. For analysis and visualisation, maps are made by astronomers with a pixelisation and binning chosen to suit the use case. FITS maps, using either a WCS or a HEALPix format, are commonly used.
An effort has been made over the past few years to define a data model and format for maps in gamma-ray astronomy (GADF; https://gamma-astro-data-formats.readthedocs.io/en/latest/skymaps/) and a prototype implementation in gammapy.maps (https://docs.gammapy.org/0.10/maps) is currently under development. Maps represent pixelized data structures with at least two spatial and an arbitrary number of non-spatial dimensions such as energy, time, event class (or any possible user-defined dimension). This is sufficiently general to act as a container for various objects like sky images and spatially dependent instrument response functions.

In this talk, I will briefly introduce GADF and its implementation within gammapy. The goal is to describe what exists, and to trigger discussion and collaboration both on data format and code aspects to reach a solution that can support many use cases from current and planned telescopes.

Speaker: Dr Atreyee SINHA (APC, IN2P3/CNRS)

maps_AS.pdf

12 Tuesday lightning talks

• Andrea Tramacere - ASTErIsM (poster)
• Felix Kunzweiler - Rapid Fermi-GBM GRB localization with BALROG (poster)
• Björn Biltzinger - Physically Motivated Background Model for GBM (poster)
• Jeremy Perkins - BurstCube
• Thomas Robitaille - Data visualization with glue

BurstCube pyGamma19.pdf

pygamma19_andrea_tramacere_asterism (Google slides)

pygamma19_andrea_tramacere_asterism.pdf

Thomas Robitaille - Glue. Demo, no slides. Link is to https://glueviz.org/

Toby Burnett - An analysis framework to create web documents

13:00 Lunch

Tuesday: Afternoon 1

13 fermitools and fermipy

I will summarize two software suites used for analysis of Fermi-LAT data: the fermitools (formerly ScienceTools), and the fermipy python-based software package.

Speaker: Dr Eric Charles (SLAC)

binned_like.pdf

Gammapy_Fermipy_1903.pdf

Gammapy_Fermipy_1903.pptx

HAI_GammaAnalysis.pdf

14 Fermi all-sky gamma-ray analysis with pointlike

The Fermi-LAT instrument differs in a fundamental way from its predecessors, particularly EGRET, in having a much larger field of view and energy range. With also having a much larger effective area, the data volume is also substantially larger. The design of pointlike took these factors into account in contrast to the more standard software tools based on the EGRET experience. I will explain this, and discuss the object-oriented design. While the basic parameter optimization is for local patches of sky, an important extension is an iterative all-sky analysis.

Speaker: Prof. Toby Burnett (University of Washington)

Fermi analysis with pointlike.pdf

Fermi analysis with pointlike.pptx

15 GSPEC - Gamma-ray Spectroscopy Analysis Package

The Fermi Gamma-ray Burst Monitor (GBM), with its broad energy range and instantaneous coverage of 2/3 of the sky, observes a wide variety of flaring or transient phenomena. These capabilities make it ideally suited for the search of transient events and to contribute to the new era of multi-messenger astrophysics. The GBM team has recently released the GSPEC analysis package, a modern GUI-based tool written in Python that uses XSPEC under the hood to perform spectral analysis of GBM data. While the initial release of the GSPEC is primary focused on GRB spectroscopy via the interactive GUI, work is underway to develop a full application programming interface (API) to facilitate the analysis, manipulation, and visualization of GBM data. The final GSPEC API will serve as an analog to the Fermipy tools developed for the Fermi Large Area Telescope (LAT).

Speaker: Dr Daniel Kocevski (NASA MSFC)

GSPEC.pdf

16:00 Coffee

Tuesday: Afternoon 2

Wednesday, 20 March

09:00 Coffee

Wednesday: Morning 1

16 Python-based Frameworks for KM3NeT

This talk focuses on three frameworks developed by the KM3NeT collaboration: KM3Pipe, aanet and OrcaNet.
KM3Pipe is a Python-based pipeline framework which is used to modularise different kinds of processes and workflows like data analysis, detector monitoring and ML training in the KM3NeT neutrino telescope experiment. Although it contains many implementations of project specific data formats and API, in its core it provides a generic and lightweight pipeline mechanism which also includes logging and basic profiling. One of its key features is the HDF5 file support which is tightly integrated with a handful of classes.
KM3NeT's offline data format is based on ROOT. The C++ data structures of the aanet framework have been designed with Python in mind from the start. Via the PyROOT binding, seamless interplay between C++ and Python code is possible, using the former for high-performance computation and the latter for high-level data analysis. Facilities also exist to the use the C++ data structures with numpy with minimal overhead.
OrcaNet is a deep learning framework based on Keras in order to simplify the training process of multiple neural networks for astroparticle physics. It incorporates automated logging, plotting and validating during the training as well as saving and continuing the training process.

Speaker: Mr Tamas Gal (KM3NeT/ECAP)

Python-based Frameworks for KM3NeT.pdf

17 ctapipe: low-level data processing for CTA

I will present the history, general design considerations and challenges related to the prototype data processing framework ctapipe. The purpose of creating ctapipe was to provide an API and standard algorithms for creating low-level (reconstruction level) data processing pipelines for Atmospheric Cherenkov Telescopes---specifically for CTA, but also supporting other existing instruments. The goals were to make it lightweight and user-friendly to support developers of varying skill level, and at the same time to allow the flexibility of design such that it might be adapted to run on anything from a user's Jupyter notebook, to batch-based systems or more complex "big-data" frameworks. The result is a fast-evolving python-based system with optional interfaces with custom high-performance computing libraries.

Speaker: Karl Kosack (CEA Saclay)

kosack-ctapipe-pygamma19.pdf

11:00 Coffee

Wednesday: Morning 2

18 Towards high-level data analysis with open-source tools in H.E.S.S.

An overview of past and current efforts to establish high-level data analysis with open-source tools (i.e. Gammapy or ctools) for the H.E.S.S. experiment will be presented. A focus will be given to ongoing work about the validation of these tools on public H.E.S.S. data (see https://www.mpi-hd.mpg.de/hfm/HESS/pages/dl3-dr1), as well as to the exploration of a 3D likelihood analysis technique. The latter requires a model template for the residual hadronic background; an attempt at the creation of such a model from archival H.E.S.S. data will be outlined.

Speaker: Lars Mohrmann (Universität Erlangen-Nürnberg)

hess_ost.pdf

19 DL3 data conversion in MAGIC

MAGIC is one of the current Imaging Atmospheric Cherenkov Telescope (IACT) located at the Roque of los Muchachos Observatory on La Palma, one of the Canary Islands. It started to operate in 2003 and is still currently running. Since 2009, MAGIC operates in a stereoscopic mode with two telescopes. A Major hardware upgrade was performed in 2012 for both camera. I will present the status on the conversion of MAGIC data since this upgrade in the high level analysis data format defined for the future very high energy observatory CTA. This open format, named DL3, is based on the FITS format and will contain a list of reconstructed gamma-ray events for each observation as well as the associated instrument response function. The discussion on this format is still open. In order to test it as well as the new open high level analysis tools, it is necessary to convert the current IACT data in this DL3 format.

I will report on the data legacy planed in the DL3 format for MAGIC data and in particular on the production of full enclosure (with no cut applied on the event direction) and offset dependent field of view instrument response function. Both are needed for morphological and 3D analysis that are a crucial point for the high level analysis software developed for CTA. I will show out first spectral results on Crab Nebula observations with Gammapy and finish by explaining our plan to produce a background model.

Speaker: Lea Jouvin (IFAE, Barcelona)

presentation.pdf

20 Status of VERITAS DL3 production

Two independent analysis chains have been classically used for the analysis of VERITAS data. These two branches start from independent low-level calibration and reach all the way to high-level science products such as spectra, skymaps, etc. With the objective of joining the effort of producing a joint multi-instrument Crab spectrum, VERITAS started exporting its high-level data products to the open high-level gamma-ray data format (DL3). This format will be ideal for the future legacy products of VERITAS. Here, I will present the current status of both analysis chains to export their data products to this format.

Speaker: Gernot Maier

VTS_DL3_20190320.pdf

13:00 Lunch

Wednesday: Afternoon 1

21 GammaLib and ctools: A Framework for the Analysis of Astronomical Gamma-ray Datasets

ctools is an open source data analysis package targeted towards the upcoming Cherenkov Telescope Array. The tools employ both a three-dimensional likelihood analysis similar to that used in the Fermi Science Tools, and a standard On/Off analysis typical of current generation atmospheric Cherenkov telescopes. Built on top of the GammaLib software, ctools is designed to be easily adapted for use with data from past, present, and future high energy photon-counting detectors. Already, the tools have been applied to H.E.S.S., Fermi-LAT, and COMPTEL data as well as simulated CTA observations. Data from multiple instruments can also be fit simultaneously, an important feature for understanding astrophysical processes across a broad energy range. In terms of the software, GammaLib exists primarily in C++ to optimize computation speed. Python wrappers are used to provide a user-friendly interface to the GammaLib classes. ctools consists of both Python modules and C++ classes that can be accessed via Python wrappers. At a higher level, the individual tools are also designed to be run directly from the command-line. This modular aspect of ctools means it can adapt to suit a range of different analyses workflows. This contribution will focus on the design and development principles behind the ctools and GammaLib software. A demonstration of the capabilities of ctools will also be presented using test data recently released by the H.E.S.S. collaboration.

Speaker: Josh Cardenzana

ctools_gammalib_PyGamma-201903.pdf

22 Gammapy - A Python Package for Gamma-Ray Astronomy

Gammapy is a community-developed, open-source Python package for gamma-ray astronomy and prototype software for the Cherenkov Telescope Array (CTA) science tools. It is built on Numpy, Scipy and Astropy as core dependencies and uses open, FITS based data formats. The development of Gammapy started ~5 years ago as a loose collection of Python library code, used for the analysis of H.E.S.S. Galactic Plane Survey data. Since this time, the range of features as well as the number of contributors and users has grown considerably. Up to now Gammapy has reached the status of alpha software and has been successfully used for first science publications and analysis of simulated CTA data. However, before Gammapy can be proposed as a complete solution for the future CTA science tools, required features have to be added and a more stable and uniform API has to be developed. In my contribution to PyGamma19 I will present in detail the current status of the package on behalf of the Gammapy team and our proposed roadmap towards a Gammapy 1.0 release. I will also share our past development experiences, present current challenges we are faced with and outline future ideas to continue the collaboration with other astronomical Python based software projects.

Speaker: Axel Donath (MPIK)

gammapy-pygamma-2019.key

gammapy-pygamma-2019.pdf

gammapy-pygamma-2019.pptx

16:00 Coffee

Wednesday: Afternoon 2

23 3ML: The Multi-Mission Maximum Likelihood Framework

As the multi-messenger era is now fully active, it is crucial that the community has a framework within which to analyze data from multiple messengers, wavelengths, and instruments in a statistically robust, common way. 3ML (https://threeml.readthedocs.io) provides an abstract, plugin-based data interface for instruments to combine analysis through each instrument's own unique likelihood. As a Python-based tool, users and instrument teams can create or use existing plugins to interface their data to a plethora of Bayesian and optimization packages in a uniform way. Analysis results are reported and stored in portable file formats that allow for the sharing and replication of results in a way that provides observers to produce robust scientific results that the community can interpret. 3ML currently supports via standard plugins many ground and space-based observatories as well as being the primary analysis tool for some collaborations. In my talk, I would like to demonstrate the capabilities and philosophy of 3ML as well as encourage instrument teams to join in the development of new plugins for a variety of instruments.

Speaker: Dr J. Michael Burgess (MPE)

pygamma_talk.pdf

24 Plenary Discussion

Plenary discussion on gamma-ray data, IRFs and science tools

19:30 Conference Dinner

Thursday, 21 March

09:00 Coffee

Thursday: Morning 1

25 Archiving data from a software telescope

In the context of Cherenkov astronomy, the data processing stages imply both assumptions and comparison to dedicated simulation. Those data can be misleading if not documented. This has implications on the format specification for the data that will be exposed to be used in relation with other frequencies for modeling.

Speaker: Catherine Boisson (Observatoire de Paris)

PyGamma19_CB.pdf

26 Provenance for multi-messenger astronomy

Provenance information in astronomy is important to enable scientists to trace back the origin of a dataset, a document or a device, learn about the people and organizations involved in a project and assess the quality as well as the usefulness of the dataset, document or device for their scientific work. Current efforts to model the Provenance information in Astronomy led to the development of tools, in particular in the context of the development of CTA (Cherenkov Telescope Array) and its python based Pipeline framework.
Astronomical observatory such as CTA are designed to produce data that will be publicly released to a large community of scientists, with strong requirements to ensure data quality, reliability and trustworthiness. Among those requirements, traceability and reproducibility of the data products have to be included early in the development of large projects. In the case of multi-messenger astronomy, the higher complexity of the data acquisition and processing reinforces the need of detailed provenance information.

Speaker: Mathieu Servillat (LUTH - Observatoire de Paris)

2019-03-21_servillat_provenance.pdf

27 A legacy portal of MAGIC published results

Since the begin of its operation in 2003, the MAGIC telescopes collected data from more than 60 TeV emitters. The collaboration distributes public FITS files with high-level data such as spectral energy distributions, light curves and skymaps for every published result. Here we report on the efforts to complement this products with more ample information (data quality, fit models, etc) and with different serialisation formats, in order to make them accessible to a wide community of users and softwares. These high-level products will be offered to the community in a legacy MAGIC data portal, under development. The status of advances will be reported.

Speaker: Michele Doro (University of Padua and INFN Padua)

doro_2019_portale.pdf

28 Plenary Discussion

11:00 Coffee

Thursday: Morning 2

29 JetSeT: a C/Python framework for numerical modeling of acceleration/radiation processes in relativistic jets and SED fitting

JetSeT is an open source Python framework with a C numerical engine, to reproduce radiative and accelerative processes acting in relativistic jets, allowing to fit the numerical models to observed data(https://jetset.readthedocs.io/en/latest/ https://github.com/andreatramacere/jetset)

The main features of this framework are:

1. handling observed data (rebinning, definition of data sets, etc...)
2. definition of complex radiative scenario SSC/EC and IC against CMB
3. Constraining of the model in the pre-fitting stage, based on accurate and already published phenomenological trends. In particular, starting from phenomenological parameters, such as spectral indices, peak fluxes and frequencies, and spectral curvatures, that the code evaluates automatically, the pre-fitting algorithm is able to provide a good starting model, following the phenomenological trends that I have implemented.
4. possibility to add template models, such as galaxies, or Big Blu Bumps
5. fitting of multiwavelength SEDs using both frequentist approach (iminuit) and MCMC sampling (emcee)
6. reproduction of the temporal evolution of the plasma under the effect of radiative and accelerative processes, both first order and second order (stochastic acceleration) processes.

I will discuss the design and the status of the project, I will try to through all the items above, in particular item 3), that as far as I know, is missing in similar packages.

Speaker: Dr Andrea Tramacere (Université de Genève)

Tramacere_PyGamm19_JetSeT.pdf

30 GAMERA toolkit, more than a Kaiju

GAMERA is a toolbox developed at MPIK for the modelling of gamma-ray emission from fundamental processes in relativistic astrophysics. Beside the emission in steady states, it allows the possibility to compute the time evolution of particle distributions in different astrophysical environments and the consequent photon spectrum detectable from an observer.
Written in C++ and wrapped in Python, it makes use of user friendliness and efficiency in the calculation. Here I will introduce the code and its basic functionalities and the development work that is in progress to enhance its capabilities. The package is available here: https://github.com/JoachimHahn/GAMERA

Speaker: Carlo Romoli (MPI-K)

gamera_pygamma_win1.pdf

31 Thursday lightning talks

• Tamas Gal - Julia - feels like Python, runs like C
• Toby Burnett - A framework to create Web documents from Python code
• Jeremy Perkins - Future Gamma-ray Missions of Various Shapes (mostly square) and Sizes
• Eric Charles - Clever tricks for fast Fermi likelihood
• Francesco Berlato - Improved GBM GRB localizations with BALROG
• Simon Steinmaßl - Fitting a Light Curve in 3ML with Template Models
• Eric Charles - How to build a pyromanic guitar

pygamma19_eric_charles_fermi_tricks.pdf

pygamma19_eric_charles_poof_lightning.pdf

pygamma19_eric_charles_poof_lightning.pptx

pygamma19_francesco_berlato_balrog.pdf

pygamma19_jeremy_perkins_future_missions.pdf

pygamma19_simon_3ml_light_curve.pdf

Tamas Gal - Julia lightning talk (screen recording)

13:00 Lunch

Thursday: Afternoon 1

32 Python in High-Energy Physics

Python is now the dominant language in scientific computing. Especially astroparticle physics and astronomy experiments have embraced Python enthusiastically, for example, the IceCube Neutrino Observatory. CERN experiments are also moving analysis steadily towards Python. ROOT is the foundational library in many HEP experiments, so I will give a brief summary on current developments in ROOT. ROOT-less analysis in Python is now also possible through the work of the Scikit-HEP project, which currently builds a complete Python stack for HEP analyses. The project has already contributed several foundational libraries, notably awkward-array, uproot, and iminuit. uproot allows one to read ROOT files in pure Python and is faster than the native C++ code. I will present these core libraries and their potential. As an outlook, I will speak about our efforts to bring a standardized object-orient fast histogram package to Python based on Boost::Histogram in C++.

Speaker: Dr Hans Peter Dembinski (Max-Planck-Institute for Nuclear Physics, Heidelberg)

Python in High-Energy Physics (HTML slideshow)

Python in High-Energy Physics.pdf

33 High performance computing with Python

I will present examples of python code optimization. First we will focus on wrapped C++ code to speed up significantly analysis. Secondly, I will talk about function minimization with PyTorch and how it can be used through two examples: likelihood minimization on GPU for ImPACT optimization and deep learning with the gamma-learn project.

Speaker: Dr Florian Gaté (LAPP - CNRS)

PyGamma19_HPC.pdf

34 Numba: Speeding up Python

An introduction to Numba, a Python package which compiles Python code on-the-fly to produce efficient machine code, potentially providing huge improvements in execution time.

Speaker: Jason Watson (University of Oxford)

190321_watson_pygamma19.pdf

35 Plenary Discussion

What are good choices / pros and cons of the various high-performance computing options in Python for our codes?

16:00 Coffee

Thursday: Afternoon 2

36 PINT - Pulsar timing with Python

https://github.com/nanograv/PINT

Speaker: Jing Luo

PINT__PyGamma19_.pdf

37 JWST Data Analysis Tools: Open Development of Community Software

Speaker: Erik Tollerud (STScI)

Tollerud_PyGamma19.key

Tollerud_PyGamma19.pdf

Tollerud_PyGamma19.pptx

38 The Advanced Scientific Data Format

Speaker: Perry Greenfield (STScI)

pygamma2019_asdf.pdf

39 Sherpa overview

Remote presentation

Speaker: Aneta Siemiginowska (Chandra X-ray Center)

Sherpa_PyGamma19_aneta.pdf

40 Sherpa technical notes

Speaker: Omar Laurino (Chandra X-ray Center)

Gammapy19_omar.pdf

Friday, 22 March

09:00 Coffee

Friday: Morning 1

41 Porting legacy software packages to the Conda Package Manager

The Fermi Science Tools are a software suite provided by the Fermi Science Support Center (FSSC) to facilitate analysis of data gathered by the Fermi Gamma-Ray Space Telescope. In the past the software was distributed via compiled binaries or compile-able source code downloads from the FSSC’s website. While this system was effective, it relied on a large amount of developer hours from the FSSC in addition to the original time contributed by the code’s authors to compile, debug, test, and package the code on all supported systems. In order to streamline this process and increase ease of use the ScienceTools were ported to the Conda Package Manager and distributed via a channel controlled by the FSSC. This has allowed for the use of Continuous Integration systems, greater ease of installation, and faster turn-around time on software updates. In this talk I will be discussing the problems we encountered making this transition with a legacy software package, detailing the benefits obtained, and presenting lessons learned in the process.

Speaker: Joseph Asercion (Fermi Science Support Center)

Asercion-Conda-PyGamma19.pdf

42 Jupyter notebooks on steroids

Jupyter notebooks have arrived to stay as a means to document the scientific analysis protocol, as well as to provide executable recipes shared seamlessly among the community. This has triggered the rise of a plethora of complementary tools and services associated to them. This talk will cover different possibilities to use Jupyter notebooks and JupyterLab interface. We will start with the description of their basic functionalities, as well as functionality extensions not widely known by the community. We will describe how to take advantage of their cross-language capabilities to enhance collaborative work, and also use them as complementary assets in the paper publication process to provide reproducibility of the results. Other aspects on how to deal with modularity and scalability of long complex notebooks will be covered, and we will see several platforms for rendering and execution others then the browser and the local desktop. We will finish on how they are actually being used together with Docker and Binder as part of the versioned executable documentation of a project like Gammapy.

Speaker: Mr Jose Enrique Ruiz (IAA - CSIC)

JupyterSteroids.pdf

JupyterSteroids.pptx

11:00 Coffee

Friday: Morning 2

43 Summary and Discussion

PyGamma19 summary and group discussion

Speaker: Christoph Deil (MPI for Nuclear Physics, Heidelberg)

2019-03-22_PyGamma19_Summary_Discussion.pdf

13:00 Lunch