Conveners
Distributed Processing and Analysis: Monday
- Jerome Lauret (BNL)
Distributed Processing and Analysis: Monday
- Pablo Saiz (CERN)
Distributed Processing and Analysis: Tuesday
- Fons Rademakers (CERN)
Distributed Processing and Analysis: Tuesday
- Patricia Mendez Lorenzo (CERN)
Distributed Processing and Analysis: Thursday
- Joel Snow (Langston University)
Distributed Processing and Analysis: Thursday
- Fabrizio Furano (CERN)
-
Dr Jakub Moscicki (CERN IT/GS), Dr Patricia Mendez Lorenzo (CERN IT/GS)23/03/2009, 14:00Distributed Processing and AnalysisoralRecently a growing number of various applications have been quickly and successfully enabled on the Grid by the CERN Grid application support team. This allowed the applications to achieve and publish large-scale results in short time which otherwise would not be possible. The examples of successful Grid applications include the medical and particle physics simulation (Geant4, Garfield),...Go to contribution page
-
Valentin Kuznetsov (Cornell University)23/03/2009, 14:20Distributed Processing and AnalysisoralThe CMS experiment has a distributed computing model, supporting thousands of physicists at hundreds of sites around the world. While this is a suitable solution for "day to day" work in the LHC era there are edge use-cases that Grid solutions do not satisfy. Occasionally it is desirable to have direct access to a file on a users desktop or laptop; for code development, debugging or examining...Go to contribution page
-
Dr Daniel van der Ster (CERN)23/03/2009, 14:40Distributed Processing and AnalysisoralGanga has been widely used for several years in Atlas, LHCb and a handful of other communities in the context of the EGEE project. Ganga provides a simple yet powerful interface for submitting and managing jobs to a variety of computing backends. The tool helps users configuring applications and keeping track of their work. With the major release of version 5 in summer 2008, Ganga's main...Go to contribution page
-
Dr Douglas Smith (STANFORD LINEAR ACCELERATOR CENTER)23/03/2009, 15:00Distributed Processing and AnalysisoralThe Babar experiment produced one of the largest datasets in high energy physics. To provide for many different concurrent analyses the data is skimmed into many data streams before analysis can begin, multiplying the size of the dataset both in terms of bytes and number of files. As a large scale problem of job management and data control, the Babar Task Manager system was...Go to contribution page
-
Dr Fabrizio Furano (Conseil Europeen Recherche Nucl. (CERN))23/03/2009, 15:20Distributed Processing and AnalysisoralThe Scalla/Xrootd software suite is a set of tools and suggested methods useful to build scalable, fault tolerant and high performance storage systems for POSIX-like data access. One of the most important recent development efforts is to implement technologies able to deal with the characteristics of Wide Area Networks, and find solutions in order to allow data analysis applications to...Go to contribution page
-
Dr Alexei Klimentov (BNL)23/03/2009, 15:40Distributed Processing and AnalysisoralWe present our experience with distributed reprocessing of the LHC beam and cosmic ray data taken with the ATLAS detector during 2008/2009. Raw data were distributed from CERN to ATLAS Tier-1 centers, reprocessed and validated. The reconstructed data were consolidated at CERN and ten WLCG ATLAS Tier-1 centers and made available for physics analysis. The reprocessing was done...Go to contribution page
-
Sophie Lemaitre (CERN)23/03/2009, 16:30Distributed Processing and AnalysisoralOne of the current problem areas for sustainable WLCG operations is in the area of data management and data transfer. The systems involved (e.g. Castor, dCache, DPM, FTS, gridFTP, OPN network) are rather complex and have multiple layers - failures can and do occur in any layer and due to the diversity of systems involved, the differences in the information they have available and their...Go to contribution page
-
Mr Levente HAJDU (BROOKHAVEN NATIONAL LABORATORY)23/03/2009, 16:50Distributed Processing and AnalysisoralProcessing datasets on the order of tens of terabytes is an onerous task, faced by production coordinators everywhere. Users solicit data productions and, especially for simulation data, the vast amount of parameters (and sometime incomplete requests) point at the need for a tracking, control and archiving all requests made so a coordinated handling could be made by the production team. With...Go to contribution page
-
Norbert Neumeister (Purdue University)23/03/2009, 17:10Distributed Processing and AnalysisoralWe present a Web portal for CMS Grid submission and management. Grid portals can deliver complex grid solutions to users without the need to download, install and maintain specialized software, or worrying about setting up site-specific components. The goal is to reduce the complexity of the user grid experience and to bring the full power of the grid to physicists engaged in LHC analysis...Go to contribution page
-
Mr Marco Meoni (CERN)23/03/2009, 17:30Distributed Processing and AnalysisoralThe ALICE experiment at CERN LHC is intensively using a PROOF cluster for fast analysis and reconstruction. The current system (CAF - CERN Analysis Facility) consists of some 120 CPU cores and about 45 TB of local space. One of the most important aspects of the data analysis on the CAF is the speed with which it can be carried out. Fast feedback on the collected data can be obtained, which...Go to contribution page
-
Dr James Letts (Department of Physics-Univ. of California at San Diego (UCSD))23/03/2009, 17:50Distributed Processing and AnalysisoralDuring normal data taking CMS expects to support potentially as many as 2000 analysis users. In 2008 there were more than 800 individuals who submitted a remote analysis job to the CMS computing infrastructure. The bulk of these users will be supported at the over 40 CMS Tier-2 centers. Supporting a globally distributed community of users on a globally distributed set of computing clusters is...Go to contribution page
-
Dr Douglas Smith (STANFORD LINEAR ACCELERATOR CENTER)23/03/2009, 18:10Distributed Processing and AnalysisoralThe Babar experiment has been running at the SLAC National Accelerator Laboratory for the past nine years, and has measured 500 fb-1 of data. The final data run for the experiment finished in April 2008. Once the data was finished the final processing of all Babar data was started. This was the largest computing production effort in the history of Babar, including a reprocessing of...Go to contribution page
-
Mrs Andrew Hanushevsky (SLAC National Accelerator Laboratory)24/03/2009, 14:00Distributed Processing and AnalysisoralScalla (also known as xrootd) is quickly becoming a significant part of LHC data analysis as a stand-alone clustered data server (US Atlas T2 and CERN Analysis Farm), globally clustered data sharing framework (ALICE), and an integral part of PROOF-base analysis (multiple experiments). Until recently, xrootd did not fit well in the LHC Grid infrastructure as a Storage Element (SE) largely...Go to contribution page
-
Dr Sergey Panitkin (Department of Physics - Brookhaven National Laboratory (BNL))24/03/2009, 14:20Distributed Processing and AnalysisoralThe Parallel ROOT Facility - PROOF is a distributed analysis system which allows to exploit inherent event level parallelism of high energy physics data. PROOF can be configured to work with centralized storage systems, but it is especially effective together with distributed local storage systems - like Xrootd, when data are distributed over computing nodes. It works efficiently on...Go to contribution page
-
Lassi Tuura (Northeastern University)24/03/2009, 14:40Distributed Processing and AnalysisoralIn the last two years the CMS experiment has commissioned a full end to end data quality monitoring system in tandem with progress in the detector commissioning. We present the data quality monitoring and certification systems in place, from online data taking to delivering certified data sets for physics analyses, release validation and offline re-reconstruction activities at Tier-1s. We...Go to contribution page
-
Dr Stuart Paterson (CERN)24/03/2009, 15:00Distributed Processing and AnalysisoralDIRAC, the LHCb community Grid solution, uses generic pilot jobs to obtain a virtual pool of resources for the VO community. In this way agents can request the highest priority user or production jobs from a central task queue and VO policies can be applied with full knowledge of current and previous activities. In this paper the performance of the DIRAC WMS will be presented with emphasis...Go to contribution page
-
Mr Andrei Gheata (CERN/ISS)24/03/2009, 15:20Distributed Processing and AnalysisoralALICE offline group has developed a set of tools that do formalize data access patterns and impose certain rules on how individual data analysis modules have to be structured in order to maximize the data processing efficiency at the whole collaboration scale. The ALICE analysis framework was developed and extensively tested on MC reconstructed data during the last 2 years in the ALICE...Go to contribution page
-
Dr Giacinto Donvito (INFN-Bari)24/03/2009, 16:30Distributed Processing and AnalysisoralThe Job Submitting Tool provides a solution for the submission of a large number of jobs to the grid in an unattended way. Indeed the tool is able to manage the grid submission, bookkeeping and resubmission of failed jobs . It also allows the monitor in real time of the status of each job using the same framework. The key elements of this tool are: A Relational Db that contains all the...Go to contribution page
-
Gerardo GANIS (CERN)24/03/2009, 16:50Distributed Processing and AnalysisoralPROOF-Lite is an implementation of the Parallel ROOT Facility (PROOF) optimized for many-core machines. It gives ROOT users a straight-forward way to exploit the many-cores by using them all in parallel for a data analysis or generic computing task controlled via the ROOT TSelector mechanism. PROOF-Lite is, as the name suggests, a lite version of PROOF, where the multi-tier...Go to contribution page
-
Dr Sanjay Padhi (UCSD)24/03/2009, 17:10Distributed Processing and AnalysisoralWith the evolution of various grid federations, the Condor glide-ins represent a key feature in providing a homogeneous pool of resources using late-binding technology. The CMS collaboration uses the glide-in based Workload Management System, glideinWMS, for production (ProdAgent) and distributed analysis (CRAB) of the data. The Condor glide-in daemons traverse to the worker nodes,...Go to contribution page
-
Dr Dantong Yu (BROOKHAVEN NATIONAL LABORATORY)24/03/2009, 17:30Distributed Processing and AnalysisoralPanDA, ATLAS Production and Distributed Analysis framework, has been identified as one of the most important services provided by the ATLAS Tier 1 facility at Brookhaven National Laboratory (BNL), and enhanced to what is now a 24x7x365 production system. During this period, PanDA has remained under active development for additional functionalities and bug fix, and processing requirements have...Go to contribution page
-
Johannes Elmsheuser (Ludwig-Maximilians-Universitรคt Mรผnchen)24/03/2009, 17:50Distributed Processing and AnalysisoralThe distributed data analysis using Grid resources is one of the fundamental applications in high energy physics to be addressed and realized before the start of LHC data taking. The needs to manage the resources are very high. In every experiment up to a thousand physicist will be submitting analysis jobs into the Grid. Appropriate user interfaces and helper applications have to be...Go to contribution page
-
Bjoern Hallvard Samset (Fysisk institutt - University of Oslo)24/03/2009, 18:10Distributed Processing and AnalysisoralA significant amount of the computing resources available to the ATLAS experiment at the LHC are connected via the ARC grid middleware. ATLAS ARC-enabled resources, which consist of both major computing centers at Tier-1 level and lesser, local clusters at Tier-2 and 3 level, have shown excellent performance running heavy Monte Carlo (MC) production for the experiment. However, with the...Go to contribution page
-
Dr Guido NEGRI (CERN)26/03/2009, 14:00Distributed Processing and AnalysisoralWithin the ATLAS hierarchical, multi-tier computing infrastructure, the Tier-0 centre at CERN is mainly responsible for prompt processing of the raw data coming from the online DAQ system, to archive the raw and derived data on tape, to register the data with the relevant catalogues and to distribute them to the associated Tier-1 centres. The Tier-0 is already fully functional. It has...Go to contribution page
-
Fabrizio Furano (Conseil Europeen Recherche Nucl. (CERN))26/03/2009, 14:20Distributed Processing and AnalysisoralPerformance, reliability and scalability in data access are key issues in the context of Grid computing and High Energy Physics (HEP) data analysis. We present the technical details and the results of a large scale validation and performance measurement achieved at the INFN Tier1, the central computing facility of the Italian National Institute for Nuclear Research (INFN). The aim of this work...Go to contribution page
-
Dr David Mason (FNAL)26/03/2009, 14:40Distributed Processing and AnalysisoralCMS' infrastructure to process, store and analyze data is based on worldwide distributed tiers of computing resources. Monitoring and trouble shooting of all parts of the computing infrastructure, and importantly the experiment specific data flows and workflows running on this infrastructure, is essential to guarantee timely delivery of processed data to the physicists. This is especially...Go to contribution page
-
Dr Andrew Maier (CERN)26/03/2009, 15:00Distributed Processing and AnalysisoralGanga (http://cern.ch/ganga) is a job-management tool that offers a simple, efficient and consistent user experience in a variety of heterogeneous environments: from local clusters to global Grid systems. Experiment specific plugins allow Ganga to be customised for each experiment. This paper will describe these LHCb plugins of Ganga. For LHCb users, Ganga is the job submission tool of choice...Go to contribution page
-
Daniel Colin Van Der Ster (Conseil Europeen Recherche Nucl. (CERN))26/03/2009, 15:20Distributed Processing and AnalysisoralEffective distributed user analysis requires a system which meets the demands of running arbitrary user applications on sites with varied configurations and availabilities. The challenge of tracking such a system requires a tool to monitor not only the functional statuses of each grid site, but also to perform large-scale analysis challenges on the ATLAS grids. This work presents one such...Go to contribution page
-
Pablo SAIZ (CERN)26/03/2009, 15:40Distributed Processing and AnalysisoralWLCG relies on the SAM (Service Availability Monitoring) infrastructure to monitor the behaviour of sites and as a powerful debugging tool. SAM is also used by individual experiments and VOs (Virtual Organisations) to submit application-specific tests to the grid. This degree of specificity implies additional requirements in terms of visualisation and manipulation of the test results provided...Go to contribution page
-
Ulrich Schwickerath (CERN)26/03/2009, 16:30Distributed Processing and AnalysisoralInstrumentation of jobs throughout its lifecycle is not obvious, as they are quite independent after being submitted, crossing multiple environments and locations until landing on a worker node. In order to measure correctly the resources used at each step, and to compare it with the view from a Fabric Infrastructure, we propose a solution using Messaging System for the Grids (MSG)...Go to contribution page
-
Prof. joel snow (Langston University)26/03/2009, 16:50Distributed Processing and AnalysisoralDZero uses a variety of resources on four continents to pursue a strategy of flexibility and automation in the generation of simulation data. This strategy provides a resilient and opportunistic system which ensures an adequate and timely supply of simulation data to support DZero's physics analyses. A mixture of facilities, dedicated and opportunistic, specialized and generic, large...Go to contribution page
-
Daniele Spiga (Universita degli Studi di Perugia & CERN)26/03/2009, 17:10Distributed Processing and AnalysisoralCMS has a distributed computing model, based on a hierarchy of tiered regional computing centres. However, the end physicist is not interested in the details of the computing model nor the complexity of the underlying infrastructure, but only to access and use efficiently and easily the remote services. The CMS Remote Analysis Builder (CRAB) is the official CMS tool that allows the access to...Go to contribution page
-
Dr Alessandra Doria (INFN Napoli)26/03/2009, 17:30Distributed Processing and AnalysisoralAn optimized use of the grid computing resources in the ATLAS experiment requires the enforcement of a mechanism of job priorities and of resource sharing among the different activities inside the ATLAS VO. This mechanism has been implemented through the VOViews publication in the information system and the fair share implementation per UNIX group in the batch system. The VOView concept...Go to contribution page
-
Mr Anar Manafov (GSI)26/03/2009, 17:50Distributed Processing and AnalysisoralโPROOF on demandโ is a set of utilities, that allows to start a PROOF cluster at user request, on a batch farm or on the Grid. It provides a plug-in based system, which allows to use different job submission frontends, such as LSF or gLite WMS. Main components of โPROOF on demandโ are the PROOFAgent and the PAConsole. PROOFAgent provides the communication layer between the xrootd...Go to contribution page
-
Ian Fisk (Fermi National Accelerator Laboratory (FNAL))26/03/2009, 18:10Distributed Processing and AnalysisoralCMS is the the process of commissioning a complex detector and a globally distributed computing model simultaneously. The represents a unique challenge for the current generation of experiments. Even at the beginning there is not sufficient analysis or organized processing resources at CERN alone. In this presentation we will discuss the unique computing challenges CMS expects to face during...Go to contribution page