Oct 10 – 14, 2016
San Francisco Marriott Marquis
America/Los_Angeles timezone

A comparison of different database technologies for the CMS AsyncStageOut transfer database

Oct 10, 2016, 3:30 PM
GG C1 (San Francisco Mariott Marquis)


San Francisco Mariott Marquis

Oral Track 2: Offline Computing Track 2: Offline Computing


Eric Vaandering (Fermi National Accelerator Lab. (US))


AsyncStageOut (ASO) is the component of the CMS distributed data analysis system (CRAB3) that manages users’ transfers in a centrally controlled way using the File Transfer System (FTS3) at CERN. It addresses a major weakness of the previous, decentralized model, namely that the transfer of the user's output data to a single remote site was part of the job execution, resulting in inefficient use of job slots and an unacceptable failure rate.

Currently ASO manages up to 600k files of various sizes per day from more than 500 users per month, spread over more than 100 site and uses a NoSQL database (CouchDB) as internal bookkeeping and as way to communicate with other CRAB3 components. Since ASO/CRAB3 were put in production in 2014, the number of transfers constantly increased up to a point where the pressure to the central CouchDB instance became critical, creating new challenges for the system scalability, performance, and monitoring. This forced a re-engineering of the ASO application to increase its scalability and lowering its operational effort.

In this contribution we present a comparison of the performance of the current NoSQL implementation and a new SQL implementation, and of how their different strength and features influenced the design choices and operational experience. We also discuss other architectural changes introduced in the system to handle the increasing load and latency in delivering the output to the user.

Primary Keyword (Mandatory) Databases

Primary authors

Justas Balcas (California Institute of Technology (US)) Marco Mascheroni (Fermi National Accelerator Lab. (US))


Diego Ciangottini (Universita e INFN, Perugia (IT)) Emilis Antanas Rupeika (Vilnius University (LT)) Eric Vaandering (Fermi National Accelerator Lab. (US)) Hassen Riahi (CERN) Jadir Marra Da Silva (UNESP - Universidade Estadual Paulista (BR)) Jose Hernandez (CIEMAT) Stefano Belforte (Universita e INFN, Trieste (IT))

Presentation materials