Sep 2 – 9, 2007
Victoria, Canada
Europe/Zurich timezone
Please book accomodation as soon as possible.

DIRAC Data Management: consistency, integrity and coherence of data

Sep 4, 2007, 11:00 AM
20m
Carson Hall C (Victoria, Canada)

Carson Hall C

Victoria, Canada

oral presentation Grid middleware and tools Grid middleware and tools

Speaker

Dr Marianne Bargiotti (European Organization for Nuclear Research (CERN))

Description

The DIRAC Data Management System (DMS) relies on both WLCG Data Management services (LCG File Catalogues, Storage Resource Managers and FTS) and LHCb specific components (Bookkeeping Metadata File Catalogue). The complexity of both the DMS and its interactions with numerous WLCG components as well as the instability of facilities concerned, has turned frequently into unexpected problems in data moving and/or data registration, preventing to have a coherent picture of datasets. Several developments in LHCb have been done in order to avoid data corruptions, data missing , data incoherence and inconsistencies among Catalogues and physical storages both through safety measures at data management level (failover mechanisms, check sums, roll back mechanisms) and extensive background checks. In this paper all the tools developed for checking data integrity and consistency will be presented, as well as a Storage Resource Checker, whose aim is to produce an up-to-date accounting of all LHCb storage usage using the LFC mirror database. Goal of this activity is the development of a generic tool suite able to categorize, analyze and systematically cure the disparate problems affecting DM in order to maintain a consistent picture of the main catalogues (Bookkeeping and LFC) and the Storage Elements.

Primary authors

Andrew Cameron Smith (European Organization for Nuclear Research (CERN)) Dr Marianne Bargiotti (European Organization for Nuclear Research (CERN))

Presentation materials