21–27 Mar 2009
Prague
Europe/Prague timezone

Distributed Analysis in ATLAS using GANGA

24 Mar 2009, 17:50
20m
Club C (Prague)

Club C

Prague

Prague Congress Centre 5. května 65, 140 00 Prague 4, Czech Republic
oral Distributed Processing and Analysis Distributed Processing and Analysis

Speaker

Johannes Elmsheuser (Ludwig-Maximilians-Universität München)

Description

The distributed data analysis using Grid resources is one of the fundamental applications in high energy physics to be addressed and realized before the start of LHC data taking. The needs to manage the resources are very high. In every experiment up to a thousand physicist will be submitting analysis jobs into the Grid. Appropriate user interfaces and helper applications have to be made available to assure that all users can use the Grid without expertise in Grid technology. These tools enlarge the number of grid users from a few production administrators to potentially all participating physicists. The GANGA job management system (http://cern.ch/ganga), developed as a common project between the ATLAS and LHCb experiments provides and integrates these kind of tools. GANGA provides a simple and consistent way of preparing, organizing and executing analysis tasks within the experiment analysis framework, implemented through a plug-in system. It allows trivial switching between running test jobs on a local batch system and running large-scale analyzes on the Grid, hiding Grid technicalities. We will be reporting on the plug-ins and our experiences of distributed data analysis using GANGA within the ATLAS experiment. Support for all grids presently used by ATLAS, namely the LCG/EGEE, NDGF/NorduGrid, and OSG/PanDA is provided. The integration and interaction with the ATLAS data management system DQ2 into GANGA is a key functionality. An intelligent job brokering is setup by using the job splitting mechanism together with dataset and file location knowledge. The brokering is aided by an automated system that regularly processes test analysis jobs at all ATLAS DQ2 supported sites. Large amounts of analysis jobs can be sent to the locations of data following the ATLAS computing model. GANGA supports amongst other things tasks of user analysis with reconstructed data and small scale production of Monte Carlo data.

Primary author

Johannes Elmsheuser (Ludwig-Maximilians-Universität München)

Co-authors

Alexander Soroko (University of Oxford) Andrew Maier (CERN) Benjamin Gaidioz (CERN) Bjorn Samset (University of Oslo) Daniel van der Ster (CERN) Frederic Brochu (University of Cambridge) Greig Cowan (University of Edinburgh) Hurng-Chun Lee (NIKHEF) Jakub Moscicki (CERN) Katarina Pajchel (University of Oslo) Mark Slater (University of Birmingham) Michael Williams (Imperial College London) Ulrik Egede (Imperial College London) Will Reece (Imperial College London)

Presentation materials