Speaker
Dr
Marianne Bargiotti
(European Organization for Nuclear Research (CERN))
Description
The DIRAC Data Management System (DMS) relies on both WLCG Data Management services
(LCG File Catalogues, Storage Resource Managers and FTS) and LHCb specific components
(Bookkeeping Metadata File Catalogue).
The complexity of both the DMS and its interactions with numerous WLCG components as
well as the instability of facilities concerned, has turned frequently into
unexpected problems in data moving and/or data registration, preventing to have a
coherent picture of datasets.
Several developments in LHCb have been done in order to avoid data corruptions, data
missing , data incoherence and inconsistencies among Catalogues and physical storages
both through safety measures at data management level (failover mechanisms, check
sums, roll back mechanisms) and extensive background checks.
In this paper all the tools developed for checking data integrity and consistency
will be presented, as well as a Storage Resource Checker, whose aim is to produce an
up-to-date accounting of all LHCb storage usage using the LFC mirror database.
Goal of this activity is the development of a generic tool suite able to categorize,
analyze and systematically cure the disparate problems affecting DM in order to
maintain a consistent picture of the main catalogues (Bookkeeping and LFC) and the
Storage Elements.
Primary authors
Andrew Cameron Smith
(European Organization for Nuclear Research (CERN))
Dr
Marianne Bargiotti
(European Organization for Nuclear Research (CERN))
