CERN Accelerating science

Talk
Title	Implementing data placement strategies for the CMS experiment based on a popularity mode
Video	Loading If you experience any problem watching the video, click the download button below Download Embed Mp4: High (600 kbps) Windows Media: Medium (480 kbps) Flash: High (753 kbps) High-resolution: MPG (2.1 GB) Copy-paste this code into your page: <iframe width="640" height="360" frameborder="0" src="https://cds.cern.ch/video/1460885?showTitle=true" allowfullscreen></iframe>
Author(s)	Giordano, Domenico (speaker) (CERN) ; Barreiro Megino, Fernando Harald (speaker) (Universidad Autonoma de Madrid (ES))
Corporate author(s)	CERN. Geneva
Imprint	2012-05-24. - Streaming video, 00:26:43:00.
Series	(Conferences) (Computing in High Energy and Nuclear Physics (CHEP) 2012)
Lecture note	on 2012-05-24T17:25:00
Subject category	Conferences
Abstract	During the first two years of data taking, the CMS experiment has collected over 20 PetaBytes of data and processed and analyzed it on the distributed, multi-tiered computing infrastructure on the WorldWide LHC Computing Grid. Given the increasing data volume that has to be stored and efficiently analyzed, it is a challenge for several LHC experiments to optimize and automate the data placement strategies in order to fully profit of the available network and storage resources and to facilitate daily computing operations. Building on previous experience acquired by ATLAS, we have developed the CMS Popularity Service that tracks file accesses and user activity on the grid and will serve as the foundation for the evolution of their data placement. A fully automated, popularity-based site-cleaning agent has been deployed in order to scan Tier2 sites that are reaching their space quota and suggest obsolete, unused data that can be safely deleted without disrupting analysis activity. Future work will be to demonstrate dynamic data placement functionality based on this popularity service and integrate it in the data and workload management systems: as a consequence the pre-placement of data will be minimized and additional replication of hot datasets will be requested automatically. This paper will give an insight into the development, validation and production process and will analyze how the framework has influenced resource optimization and daily operations in CMS.
Copyright/License	© 2012-2024 CERN
Submitted by	jd@bnl.gov