10.5 Infrastructure

10.5.3 Data Lakes

One of the directions we are actively pursuing for grid storage provided by WLCG resources is to extend the concentration of disk resources at fewer and larger, possibly federated, sites. Smaller cache- like storage would serve the more widely distributed computing resources. The reasoning is that the provisioning and operation of robust storage requires significantly more manpower than needed for a compute cluster with a cache of secondary replicas. In principle, distributed operations are simplified in this scenario, although separation of storage and computing raised their own reliability and operational challenges in the past. Results from a WLCG survey suggest the staffing requirement is perhaps not a driving factor for a typical Tier 2. Also local and regional funding tends to work against concentration of resources. Nevertheless, some sites or funding agencies have already chosen to connect computing facilities to remote storage through cache layer. ATLAS needs to prepare for a significant fraction of remote accesses.