Speaker
Dr
Stuart Wakefield
(Imperial College London)
Description
From its conception the job management system has been distributed to increase scalability and robustness. The system consists of several applications (called prodagents) which each manage Monte Carlo, reconstruction and skimming jobs on collections of sites within different Grid environments (OSG, NorduGrid?, LCG) and submission systems (GlideIn?, local batch, etc..).
Production of simulated data in CMS will take place mainly on so called Tier2s (small to medium size computing centers) resources. Approximately ~50% of the CMS Tier2 resources are allocated to running simulation jobs. While the so called Tier1s (medium to large size computing centers with high capacity tape storage systems) will be mainly used for skimming and reconstructing detector data. During the last one and a half years the system has also been adapted such that it can be configured for converting Data Acquisition (DAQ)/ High Level Trigger (HLT) output from the CMS detector to the CMS data format and manage the real time data stream from the experiment. Simultaneously the system has been upgraded to facilitate the increasing scale of the CMS production and adapting to the procedures used by its operators.
In this paper we discuss the current (high level) architecture of ProdAgent, the experience in using this system in computing challenges, feedback from these challenges, and future work including migration to a set of core libraries to facilitate convergence between the different data management projects within CMS that deal with analysis, simulation, and initial reconstruction of real data. This migration is important as it will decrease the code footprint used by these projects and increase maintainability of the code base.
Author
Dr
David Evans
(Fermilab, Batavia, IL, USA)
Co-authors
Mr
Ahmad Hassan
(CERN, Geneva, Switzerland)
Dr
Ajit Mohapatra
(Wisconsin)
Dr
David Mason
(Fermilab, Batavia, IL, USA)
Dr
Dirk Hufnagel
(CERN, Geneva, Switzerland)
Mr
Frank van Lingen
(Caltech)
Dr
Mike Miller
(MIT)
Dr
Oliver Gutsche
(Fermilab, Batavia, IL, USA)
Dr
Simon Metson
(H.H. Wills Physics Laboratory, Bristol University)
Dr
Stuart Wakefield
(Imperial College London)
