29 November 2021 to 3 December 2021
Virtual and IBS Science Culture Center, Daejeon, South Korea
Asia/Seoul timezone

Data transfer for STAR grid jobs

contribution ID 635
Not scheduled
20m
Crystal (Gather.Town)

Crystal

Gather.Town

Poster Track 1: Computing Technology for Physics Research Posters: Crystal

Speakers

Irakli Chakaberia (Lawrence Berkeley National Lab. (US)) Irakli Chakaberia (Lawrence Berkeley National Lab. (US))

Description

Solenoidal Tracker at RHIC (STAR) is a multipurpose experiment at the Relativistic Heavy Ion Collider (RHIC) with the primary goal to study formation and properties of the quark-gluon plasma. STAR is an international collaboration of member institutions and laboratories from around the world. Yearly data-taking period produces PBytes of raw data collected by the experiment. STAR primarily uses its dedicated facility at BNL to process this data, but has routinely leveraged distributed systems, both high throughput (HTC) and high performance (HPC) computing clusters, to significantly augment the processing capacity available to the experiment.

The ability to automate the efficient transfer of large data sets on reliable, scalable, and secure infrastructure is critical for any large-scale distributed processing campaign. For more than a decade, STAR computing has relied upon GridFTP with its X.509-based authentication to build such data transfer systems and integrate them into its larger production workflow. The end of support by the community for both GridFTP and the X.509 standard requires STAR to investigate other approaches to meet its distributed processing needs.

In this study we investigate two multi-purpose data distribution systems, Globus.org and XRootD, as alternatives to GridFTP. We compare both their performance and the ease by which each service is integrated into the type of secure and automated data transfer systems STAR has previously built using GridFTP. The presented approach and study may be applicable to other distributed data processing use cases beyond STAR.

Significance

Presented approach is new for the STAR experiment. It is based on direct implementation of the mentioned tools without extra layer of custom software, thus could be transferred for other similar uses directly, and discusses their performance.

Speaker time zone Compatible with America

Primary authors

Irakli Chakaberia (Lawrence Berkeley National Lab. (US)) Jeff Porter (Lawrence Berkeley National Lab. (US)) Jerome LAURET (Brookhaven National Laboratory) Michael Poat (Brookhaven National Laboratory)

Presentation materials