Jul 9 – 13, 2018
Sofia, Bulgaria
Europe/Sofia timezone

An open source data transfer toolkit for research data

Jul 10, 2018, 4:00 PM
Sofia, Bulgaria

Sofia, Bulgaria

National Culture Palace, Boulevard "Bulgaria", 1463 NDK, Sofia, Bulgaria
Poster Track 4 - Data Handling Posters


David Colling (Imperial College (GB))


Many areas of academic research are increasingly catching up with the LHC experiments
when it comes to data volumes, and just as in particle physics they require large data sets to be moved between analysis locations.
The LHC experiments have built a global e-Infrastructure in order to handle hundreds of
petabytes of data and massive compute requirements. Yet, there is nothing particle physics specific in this infrastructure. However its command line interfaces and its use of an individual’s X.509 certificate as a means of user identification can be seen as barriers by other communities. These users typically have access to multiple endpoints via site specific logins, but no X.509 certificates or any other shared authentication infrastructure.
This project aims to provide an easy to use ‘drag-and-drop’ web interface for users to transfer files between institutions that do not have a grid infrastructure in place. Consideration was given to the simplicity of deployment, by providing packaged version for common operating systems. The underlying technology. e.g. automatic generation of X.509 certificates, is hidden from the user, but leverages standard grid technologies.
We report on the development of a prototype for the UK distributed supercomputing facility (DiRAC) and the planned extension of the project to other communities.

Primary authors

David Colling (Imperial College (GB)) Simon Fayer (Imperial College (GB)) Alexander John Richards (Imperial College (GB)) Janusz Martyniak Daniela Bauer (Imperial College (GB))

Presentation materials