Speaker
Stephen Gowdy
(CERN)
Description
The work is focused on the creation and validation tests of a replica and transfers system for Computational Grids
inspired on the needs of the High Energy Physics (HEP).
Due to the high volume of data created by the HEP experiments, an efficient file and dataset replica system may play
an important role on the computing model. Data replica systems allow the creation of copies, distributed between
the different storage elements on the Grid.
In the HEP context, the data files are basically immutable. This eases the task of the replica system, because given
sufficient local storage resources any given dataset only needs to be replicated to a particular site once.
Concurrent with the advent of computational Grids, another important theme in the distributed systems area that has
also seen some significant interest is that of peer-to-peer networks (p2p). P2p networks are an important and
evolving mechanism that facilitates the use of distributed computing and storage resources by end users.
One common technique to achieve faster file downloads from possibly overloaded storage elements over congested
networks is to split the files into smaller pieces. This way, each piece can be transferred from a different
replica, in parallel or not, optimizing the moments in that the network conditions are better suited to the
transfer.
The main tasks achieved by the system are: the creation of replicas, the development of a system for replicas
transfer (RFT) and for replicas location (RLS) with a different architecture that the one provided by Globus and the
development of a system for file transfer in pieces on computational grids with interfaces for several storage
elements.
The RLS uses a p2p overlay based on the Kademlia algorithm.
Author
Mr
Marko Petek
(UERJ)
Co-authors
Prof.
Alberto Santoro
(Universidade do Estado do Rio de Janeiro (BR))
Mr
Claudio Fernando Resin Geyer
(UFRGS)
Diego Da Silva Gomes
(Universidade do Estado do Rio de Janeiro (BR))
Stephen Gowdy
(CERN)
