14-18 October 2013
Amsterdam, Beurs van Berlage
Europe/Amsterdam timezone

Data Processing for the Dark Energy Survey

15 Oct 2013, 16:51
Keurzaal (Amsterdam, Beurs van Berlage)


Amsterdam, Beurs van Berlage

Oral presentation to parallel session Distributed Processing and Data Handling B: Experiment Data Processing, Data Handling and Computing Models Distributed Processing and Data Handling B: Experiment Data Processing, Data Handling and Computing Models


Donald Petravick (U)


The Dark Energy Survey (DES) is designed to probe the origin of the accelerating universe and help uncover the nature of dark energy by measuring the 14-billion-year history of cosmic expansion with high precision. More than 120 scientists from 23 institutions in the United States, Spain, the United Kingdom, Brazil, Switzerland and and Germany are working on the project. This collaboration has built an extremely sensitive 570-Megapixel digital camera, DECam, and has mounted it on the Blanco 4-meter telescope at Cerro Tololo Inter-American Observatory high in the Chilean Andes. The survey has completed an initial season of science verification. The survey will start in September 2013 and run for 5 years. DES data are transferred by network to the National Center for Supercomputing Applications at the University of Illinois at Urbana-Champaign. The processing system there supports quick-turn-around processing for super nova science and final processing of data into catalogs. We describe the processing software system which is in place for the five year data taking period. The system is capable of processing data on mid scale super computers and the Open Science Grid. The software structure is oriented towards wrapping community codes and custom codes, in a way that provides for uniform handling and common operational characteristics for 10 processing pipelines. The system is supported by a 100TB oracle database, which is used to store object catalogs as well as extensive operational and file system meta-data. Provenance data is stored in in a uniform schema derived from the Open Provenance Model.

Primary author


Dr Adam Lyon (Fermilab) Dr Elizabeth Buckley-Geer (Fermi National Accelerator Laboratory) Ms Margaret Gelman (National Center for Supercomputing Applications) Mr Michael Johnson (National Center for Supercomputing Applications) Michael Wang Ms Michelle Gower (National Center for Supercomputing Applications) Mr Todd Tomashek (National Center for Supercomputing Applications)

Presentation Materials