Speaker
Mr
Fabio Hernandez
(IN2P3/CNRS Computing Center and IHEP Computing Center)
Description
In this contribution we present our experience building a prototype of an open source software system for operating online file repositories of extensible capacity.
Built on the well-understood client-server architecture model, the system can be used by computing centers looking at solutions for providing online storage services for their individual users. The client-side component runs on the end-user’s personal computer and provides both command-line and graphical user interfaces. It supports a deliberately limited set of operations on remote files, namely store, retrieve, organize and share remote files.
Mucura exposes the same HTTP-based standard API supported by Amazon S3 and extends it to support the certificate-based authentication mechanism used by production grid computing platforms such as WLCG. As a consequence, personal file repositories based on Mucura can be seamlessly accessed both from the user’s personal computer and from grid jobs running on the user’s behalf. This integration allows researchers to use their individual online storage space as a personal storage element conveniently managed from their personal computer.
At the core of the system there are components for managing file metadata and for secure storage of the files’ contents, implemented on top of highly available, distributed, persistent and scalable key-value stores.
We will present a detailed architectural view of the system, the status of development and the perspectives for the months to come.
This work is inspired not only by the increasing number of commercial services available nowadays to individuals for their personal storage needs (backup, file sharing, synchronization, …) such as Amazon S3, Dropbox, SugarSync, bitcasa, etc., but also by several efforts in the same area in the academic and research worlds (NASA, SDSC, etc.). We are persuaded that the level of flexibility offered to individuals by this kind of systems adds value to the day-to-day work of scientists.
Author
Mr
Fabio Hernandez
(IN2P3/CNRS Computing Center and IHEP Computing Center)
