Speaker
Valeria Bartsch
(FERMILAB / University College London)
Description
CDF has recently changed its data handling system from the DFC (Data File Catalogue)
system to the SAM (Sequential Access to Metadata) system. This change was done as a
preparation for distributed computing because SAM can handle distributed computing
and provides mechanisms which enable it to work together with GRID systems.
Experience shows that the usage of a new data handling system increases rapidly if
it incorporates as many use cases from the old system as possible and has an
interface which is similar to the old system. This is also true for the transition
of the DFC system to the SAM system.
The CDF Analysis Input and Output Modules are the preferred interfaces to access
input data and direct output data in a standard analysis job. The Input Module
invokes the data handling system in order to manage the data delivery to the job and
to handle exceptions if problems with the data handling are experienced. The change
of the data handling system has been done in such a way that it is nearly transparent
to the user but still gives the user the choice which system to use. During the
transition to the SAM data handling system methods have been tested to give reliable
access to the data in case that the underlying storage mechanism (DCache) is
failing. Another big issue of the interface is recovery of jobs which have failed.
At FNAL most of the analysis jobs are running at the CAF (Central Analysis Farm). The
CAF is basically a batch system, but has been expanded for example with the features
of authentication. Therefore it is possible to distribute the computing for CDF with
so-called DCAF systems (Decentralized CAF systems). The CAF system provides at the
beginning and the end of the job interfaces to SAM, so it can be used to
automatically recover jobs which failed. The Input Module is able to provide the
necessary interfaces to the CAF to make the recovery possible. To implement this
mechanism is good step in the direction of reliable jobs on the GRID.
Primary author
Valeria Bartsch
(FERMILAB / University College London)
Co-authors
Andrew Baranovski
(Fermilab)
Doug Benjamin
(Duke University)
Elliot Lipeles
(Univ. of California, San Diego)
Fedor Ratnikov
(University of Maryland)
Igor Sfiligoi
(Fermilab)
Krzysztof Genser
(Fermilab)
Rick St. Denis
(Glasgow university (Scotland))
SHIH-CHIEH Hsu
(Univ. of California, San Diego)
Stefan Stonjek
(Oxford university)
