Conveners
Distributed Processing and Analysis on Grids and Clouds
- Oliver Gutsche (Fermi National Accelerator Lab. (US))
Distributed Processing and Analysis on Grids and Clouds
- Johannes Elmsheuser (Ludwig-Maximilians-Univ. Muenchen (DE))
Distributed Processing and Analysis on Grids and Clouds
- Philippe Canal (Fermi National Accelerator Lab. (US))
Distributed Processing and Analysis on Grids and Clouds
- Philippe Canal (Fermi National Accelerator Lab. (US))
Distributed Processing and Analysis on Grids and Clouds
- Oliver Gutsche (Fermi National Accelerator Lab. (US))
Distributed Processing and Analysis on Grids and Clouds
- Johannes Elmsheuser (Ludwig-Maximilians-Univ. Muenchen (DE))
Distributed Processing and Analysis on Grids and Clouds
- Oliver Gutsche (Fermi National Accelerator Lab. (US))
Pablo Saiz
(CERN)
5/21/12, 1:30โฏPM
Distributed Processing and Analysis on Grids and Clouds (track 3)
Parallel
AliEn is the GRID middleware used by the ALICE collaboration. It provides all the components that are needed to manage the distributed resources. AliEn is used for all the computing workflows of the experiment: Montecarlo production, data replication and reconstruction and organixed or chaotic user analysis. Moreover, AliEn is also being used by other experiments like PANDA and CBM.
The...
Vincent Garonne
(CERN)
5/21/12, 1:55โฏPM
Distributed Processing and Analysis on Grids and Clouds (track 3)
Parallel
The ATLAS collaboration has recorded almost 5PB of RAW data since the LHC started running at the end of 2009. Together with experimental data generated from RAW and complimentary simulation data, and accounting for data replicas on the grid, a total of 74TB is currently stored in the Worldwide LHC Computing Grid by ATLAS. All of this data is managed by the ATLAS Distributed Data Management...
Dr
Stuart Wakefield
(Imperial College London)
5/21/12, 2:20โฏPM
Distributed Processing and Analysis on Grids and Clouds (track 3)
Parallel
CMS has started the process of rolling out a new workload management
system. This system is currently used for reprocessing and monte carlo
production with tests under way using it for user analysis.
It was decided to combine, as much as possible, the
production/processing, analysis and T0 codebases so as to reduce
duplicated functionality and make best use of limited developer...
Philippe Charpentier
(CERN)
5/21/12, 2:45โฏPM
Distributed Processing and Analysis on Grids and Clouds (track 3)
Parallel
The LHCb Data Management System is based on the DIRAC Grid Community Solution. LHCbDirac provides extensions to the basic DMS such as a Bookkeeping System. Datasets are defined as sets of files corresponding to a given query in the Bookkeeping system. Datasets can be manipulated by CLI tools as well as by automatic transformations (removal, replication, processing). A dynamic handling of...
Dr
Maria Girone
(CERN)
5/21/12, 3:10โฏPM
Distributed Processing and Analysis on Grids and Clouds (track 3)
Parallel
After two years of LHC data taking, processing and analysis and with numerous changes in computing technology, a number of aspects of the experimentsโ computing as well as WLCG deployment and operations need to evolve. As part of the activities of the Experiment Support group in CERNโs IT department, and reinforced by effort from the EGI-InSPIRE project, we present work aimed at common...
Andrew John Washbrook
(University of Edinburgh (GB))
5/21/12, 4:35โฏPM
Distributed Processing and Analysis on Grids and Clouds (track 3)
Parallel
AthenaMP is the multi-core implementation of the ATLAS software framework and allows the efficient sharing of memory pages between multiple threads of execution. This has now been validated for production and delivers a significant reduction on overall memory footprint with negligible CPU overhead.
Before AthenaMP can be routinely run on the LHC Computing Grid, it must be determined how the...
Dr
Jose Hernandez Calama
(Centro de Investigaciones Energ. Medioambientales y Tecn. - (ES)
5/21/12, 5:00โฏPM
Distributed Processing and Analysis on Grids and Clouds (track 3)
Parallel
Commodity hardware is going many-core. We might soon not be able to satisfy the job memory needs per core in the current single-core processing model in High Energy Physics. In addition, an ever increasing number of independent and incoherent jobs running on the same physical hardware not sharing resources might significantly affect processing performance. It will be essential to effectively...
Anar Manafov
(GSI - Helmholtzzentrum fur Schwerionenforschung GmbH (DE))
5/21/12, 5:25โฏPM
Distributed Processing and Analysis on Grids and Clouds (track 3)
Parallel
PROOF on Demand (PoD) is a tool-set, which dynamically sets up a PROOF cluster at a userโs request on any resource management system (RMS). It provides a plug-in based system, in order to use different job submission front-ends.
PoD is currently shipped with gLite, LSF, PBS (PBSPro/OpenPBS/Torque), Grid Engine (OGE/SGE), Condor, LoadLeveler, and SSH plug-ins. It makes it possible just within...
Luis Granado Cardoso
(CERN)
5/21/12, 5:50โฏPM
Distributed Processing and Analysis on Grids and Clouds (track 3)
Parallel
LHCb is one of the 4 experiments at the LHC accelerator at CERN. LHCb has approximately 1600 (8 cores) PCs for processing the High Level Trigger (HLT) during physics data acquisition. During periods when data acquisition is not required or the resources needed for data acquisition are reduced, like accelerator Machine Development (MD) periods or technical shutdowns, most of these PCs are idle...
Artem Harutyunyan
(CERN)
5/22/12, 1:30โฏPM
Distributed Processing and Analysis on Grids and Clouds (track 3)
Parallel
CernVM Co-Pilot is a framework for instantiating an ad-hoc computing infrastructure on top of distributed computing resources. Such resources include commercial computing clouds (e.g. Amazon EC2), scientific computing clouds (e.g. CERN lxcloud), as well as the machines of users participating in volunteer computing projects (e.g. BOINC). The framework consists of components that communicate...
Victor Manuel Fernandez Albor
(Universidade de Santiago de Compostela (ES)),
Victor Mendez Munoz
(Port d'Informaciรณ Cientรญfica (PIC))
5/22/12, 1:55โฏPM
Distributed Processing and Analysis on Grids and Clouds (track 3)
Parallel
The increasing availability of cloud resources is making the
scientific community to consider a choice between Grid and Cloud.
The DIRAC framework for distributed computing is an easy way
to obtain resources from both systems.
In this paper we explain the integration of DIRAC with a two
Open-source Cloud Managers, OpenNebula and CloudStack.
They are computing tools to manage the...
Fernando Harald Barreiro Megino
(CERN IT ES)
5/22/12, 2:20โฏPM
Distributed Processing and Analysis on Grids and Clouds (track 3)
Parallel
The ATLAS Computing Model was designed around the concepts of grid computing; since the start of data-taking, this model has proven very successful in the federated operation of more than one hundred Worldwide LHC Computing Grid (WLCG) sites for offline data distribution, storage, processing and analysis. However, new paradigms in computing, namely virtualization and cloud computing, present...
Oliver Oberst
(KIT - Karlsruhe Institute of Technology (DE))
5/22/12, 2:45โฏPM
Distributed Processing and Analysis on Grids and Clouds (track 3)
Parallel
The specific requirements concerning the software environment within the HEP community constrain the choice of resource providers for the outsourcing of computing infrastructure. The use of virtualization in HPC clusters and in the context of cloud resources is therefore a subject of recent developments in scientific computing.
The dynamic virtualization of worker nodes in common batch...
Mr
Alessandro Italiano
(INFN-CNAF), Dr
Giacinto Donvito
(INFN-Bari)
5/22/12, 3:10โฏPM
Distributed Processing and Analysis on Grids and Clouds (track 3)
Parallel
In this paper we present the latest developments introduced in the WNoDeS framework (http://web.infn.it/wnodes); we will in particular describe inter-cloud connectivity, support for multiple batch systems, and coexistence of virtual and real environments on a single hardware.
Specific effort has been dedicated to the work needed to deploy a "multi-sites" WNoDeS installation. The goal is to...
Thomas Kuhr
(KIT - Karlsruhe Institute of Technology (DE))
5/22/12, 4:35โฏPM
Distributed Processing and Analysis on Grids and Clouds (track 3)
Parallel
The Belle II experiment, a next-generation B factory experiment at KEK, is expected to record a two orders of magnitude larger data volume than its predecessor, the Belle experiment. The data size and rate are comparable to or more than the ones of LHC experiments and requires to change the computing model from the Belle way, where basically all computing resources were provided by KEK, to a...
Daniel Colin Van Der Ster
(CERN)
5/22/12, 4:35โฏPM
Distributed Processing and Analysis on Grids and Clouds (track 3)
Parallel
Frequent validation and stress testing of the network, storage and CPU resources of a grid site is essential to achieve high performance and reliability. HammerCloud was previously introduced with the goals of enabling VO- and site-administrators to run such tests in an automated or on-demand manner. The ATLAS, CMS and LHCb experiments have all developed VO plugins for the service and have...
Dmitry Ozerov
(Deutsches Elektronen-Synchrotron (DE)), Dr
Patrick Fuhrmann
(DESY)
5/22/12, 5:00โฏPM
Distributed Processing and Analysis on Grids and Clouds (track 3)
Parallel
One of the most crucial requirement for online storage is the fast and efficient access to data.
Although smart client side caching often compensates for discomforts like latencies and server disk congestion, spinning disks, with their limited ability to serve multi stream random access patterns, seem to be the cause of most of the observed inefficiencies.
With the appearance of the...
Mrs
Jianlin Zhu
(Central China Normal University (CN))
5/22/12, 5:00โฏPM
Distributed Processing and Analysis on Grids and Clouds (track 3)
Parallel
A Grid is a geographically distributed environment with autonomous sites that share resources collaboratively. In this context, the main issue within a Grid is encouraging site to site interactions, increasing the trust, confidence and reliability of the sites to share resources. To achieve this, the trust concept is vital component in every service transaction, and needs to be applied in the...
Dr
Stefan Lueders
(CERN)
5/22/12, 5:25โฏPM
Distributed Processing and Analysis on Grids and Clouds (track 3)
Parallel
Access protection is one of the cornerstones of security. The rule of least-privilege demands that any access to computer resources like computing services or web applications is restricted in such a way that only users with a need-to can access those resources. Usually this is done when authenticating the user asking her for something she knows, e.g. a (public) username and secret password....
Parag Mhashilkar
(Fermi National Accelerator Laboratory)
5/22/12, 5:25โฏPM
Distributed Processing and Analysis on Grids and Clouds (track 3)
Parallel
Grid computing has enabled scientific communities to effectively share computing resources distributed over many independent sites. Several such communities, or Virtual Organizations (VO), in the Open Science Grid and the European Grid Infrastructure use the glideinWMS system to run complex application work-flows. GlideinWMS is a pilot-based workload management system (WMS) that creates on...
Iwona Sakrejda,
Jeff Porter
(Lawrence Berkeley National Lab. (US))
5/22/12, 5:50โฏPM
Distributed Processing and Analysis on Grids and Clouds (track 3)
Parallel
The ALICE Grid infrastructure is based on AliEn, a lightweight open source framework built on Web Services and a Distributed Agent Model in which job agents are submitted onto a grid site to prepare the environment and pull work from a central task queue located at CERN. In the standard configuration, each ALICE grid site supports an ALICE-specific VO box as a single point of contact between...
Dr
Andrea Sciaba
(CERN),
Lothar A.T. Bauerdick
(Fermi National Accelerator Lab. (US))
5/22/12, 5:50โฏPM
Distributed Processing and Analysis on Grids and Clouds (track 3)
Parallel
The operation of the CMS computing system requires a complex monitoring system to cover all its aspects: central services, databases, the distributed computing infrastructure, production and analysis workflows, the global overview of the CMS computing activities and the related historical information. Several tools are available to provide this information, developed both inside and outside of...
Tony Cass
(CERN)
5/24/12, 1:30โฏPM
Distributed Processing and Analysis on Grids and Clouds (track 3)
Parallel
The HEPiX Virtualisation Working Group has sponsored the development of policies and technologies that permit Grid sites to safely instantiate remotely generated virtual machine images confident in the knowledge that they will be able to meet their obligations, most notably in terms of guaranteeing the accountability and traceability of any Grid Job activity at their site.
We will present...
Jeff Templon
(NIKHEF (NL))
5/24/12, 1:55โฏPM
Distributed Processing and Analysis on Grids and Clouds (track 3)
Parallel
This contribution describes a prototype grid proxy cache system developed at Nikhef, motivated by a desire to construct the first
building block of a future https-based Content Delivery Network for multiple-VO grid infrastructures. Two goals drove the project:
firstly to provide a "native view" of the grid for desktop-type users, and secondly to improve performance for physics-analysis type...
Dr
Armando Fella
(INFN Pisa)
5/24/12, 2:20โฏPM
Distributed Processing and Analysis on Grids and Clouds (track 3)
Parallel
The SuperB asymmetric energy e+e- collider and detector to be built at the newly founded Nicola Cabibbo Lab will provide a uniquely sensitive probe of New Physics in the flavor sector of the Standard Model. Studying minute effects in the heavy quark and heavy lepton sectors requires a data sample of 75 ab-1 and a luminosity target of 10^36 cm-2 s-1.
The increasing network performance also...
Dr
Balazs Konya
(Lund University (SE))
5/24/12, 2:45โฏPM
Distributed Processing and Analysis on Grids and Clouds (track 3)
Parallel
Scientific research communities have benefited recently from the increasing availability of computing and data infrastructures with unprecedented capabilities for large scale distributed initiatives. These infrastructures are largely defined and enabled by the middleware they deploy. One of the major issues in the current usage of research infrastructures is the need to use similar but often...
Mrs
Ruth Pordes
(Fermi National Accelerator Lab. (US))
5/24/12, 3:10โฏPM
Distributed Processing and Analysis on Grids and Clouds (track 3)
Parallel
As it enters adolescence the Open Science Grid (OSG) is bringing a maturing fabric of Distributed High Throughput Computing (DHTC) services that supports an expanding HEP community to an increasingly diverse spectrum of domain scientists. Working closely with researchers on campuses throughout the US and in collaboration with national cyberinfrastructure initiatives, we transform their...
Paul Millar
(Deutsches Elektronen-Synchrotron (DE))
5/24/12, 4:35โฏPM
Distributed Processing and Analysis on Grids and Clouds (track 3)
Parallel
For over a decade, dCache has been synonymous with large-capacity, fault-tolerant storage using commodity hardware that supports seamless data migration to and from tape. Over that time, it has satisfied the requirements of various demanding scientific user communities to store their data, transfer it between sites and fast, site-local access.
When the dCache project started, the focus was...
Mr
Zsolt Molnรกr
(CERN)
5/24/12, 5:00โฏPM
Distributed Processing and Analysis on Grids and Clouds (track 3)
Parallel
LHC experiments at CERN and worldwide utilize WLCG resources and middleware components to perform distributed computing tasks. One of the most important tasks is reliable file replication. It is a complex problem, suffering from transfer failures, disconnections, transfer duplication, server and network overload, differences in storage systems, etc. To address these problems, EMI and gLite...
Dr
Domenico Giordano
(CERN),
Fernando Harald Barreiro Megino
(Universidad Autonoma de Madrid (ES))
5/24/12, 5:25โฏPM
Distributed Processing and Analysis on Grids and Clouds (track 3)
Parallel
During the first two years of data taking, the CMS experiment has collected over 20 PetaBytes of data and processed and analyzed it on the distributed, multi-tiered computing infrastructure on the WorldWide LHC Computing Grid. Given the increasing data volume that has to be stored and efficiently analyzed, it is a challenge for several LHC experiments to optimize and automate the data...
Tadashi Maeno
(Brookhaven National Laboratory (US))
5/24/12, 5:50โฏPM
Distributed Processing and Analysis on Grids and Clouds (track 3)
Parallel
The PanDA Production and Distributed Analysis System is the ATLAS workload management system for processing user analysis, group analysis and production jobs.
In 2011 more than 1400 users have submitted jobs through PanDA to the ATLAS grid infrastructure. The system processes more than 2 million analysis jobs per week. Analysis jobs are routed to sites based on the availability of relevant...
