10–15 Mar 2019
Steinmatte conference center
Europe/Zurich timezone

Cross-domain Data Access System for Distributed Sites in HEP

11 Mar 2019, 18:40
20m
Steinmatte Plenary

Steinmatte Plenary

Oral Track 1: Computing Technology for Physics Research Track 1: Computing Technology for Physics Research

Speaker

Mr Qi Xu (Institute of High Energy Physics, Chinese Academy of Sciences)

Description

A large amount of data is produced by large scale scientific facilities in high energy physics (HEP) field. And distributed computing technologies has been widely used to process these data. In traditional computing model such as grid computing, computing job is usually scheduled to the sites where the input data was pre-staged in. This model will lead to some problems includ-ing low CPU utilization, inflexibility, and difficulty in highly dynamic cloud environment. The paper proposed a cross-domain data access system (CDAS), which presents one same file system view at local and the remote sites, supporting directly data access on demand. Then the computing job can run everywhere no need to know where data is located. For the moment the system has been implemented including these functionalities such as native access for remote data, quick response, data transmission and management on demand based on HTTP, data block hash and store, uniform file view and so on. The test results showed the performance was much better than tradi-tional file system on high-latency WAN.

Primary authors

Mr Qi Xu (Institute of High Energy Physics, Chinese Academy of Sciences) Zhenjing Cheng (INSTITUE OF HIGH ENERGY PHYSICS) Yaodong Cheng (IHEP) Gang CHEN (INSTITUTE OF HIGH ENERGY PHYSICS)

Presentation materials