4–8 Nov 2019
Adelaide Convention Centre
Australia/Adelaide timezone

IceCube Real-time Processing in AWS

7 Nov 2019, 12:00
15m
Riverbank R7 (Adelaide Convention Centre)

Riverbank R7

Adelaide Convention Centre

Oral Track 7 – Facilities, Clouds and Containers Track 7 – Facilities, Clouds and Containers

Speakers

Benedikt Riedel (University of Wisconsin-Madison) Benedikt Riedel (University of Wisconsin-Madison)

Description

IceCube sends out real-time alerts for neutrino events to other multi-messenger observatories around the world, including LIGO/VIRGO and electromagnetic observatories. The typical case is to send out an initial alert within one minute, then run more expensive processing to refine the direction and energy estimates and send a follow-on message. This second message has averaged 40 to 60 minutes of delay over the last two years, but is essential for pointing telescopes that have a small field of view. Recently we have moved from a local compute cluster to bursting in Amazon’s cloud to increase parallelization and decrease the follow-on message delay. This involved creating new distribution architectures for event data and collection of results, to scale beyond a few thousand workers. We will give an overview of the steps we took, how many machines we are able to scale to, and the improvement in follow-on message delay.

Consider for promotion Yes

Authors

David Schultz (University of Wisconsin-Madison) Benedikt Riedel (University of Wisconsin-Madison)

Presentation materials