Speaker
Dr
Marcin Nowak
(Brookhaven National Laboratory)
Description
In anticipation of data taking, ATLAS has undertaken a program of work
to develop an explicit state representation of the experiment's complex transient
event data model. This effort has provided both an opportunity to
consider explicitly the structure, organization, and content of the ATLAS persistent
event store before writing tens of petabytes of data (replacing simple streaming,
which uses the persistent store as a core dump of transient memory), and a locus
for support of event data model evolution, including significant refactoring,
beyond the automatic schema evolution capabilities of underlying persistence
technologies. ATLAS has encountered the need for such non-trivial
schema evolution on several occasions already.
This paper describes the state representation strategy (transient/persistent
separation) and its implementation, including both the payoffs that ATLAS
has seen (significant and sometimes surpising space and performance improvements,
the extra layer notwithstanding, and extremely general schema evolution
support) and the costs (additional and relatively pervasive additional
infrastructure development and maintenance). The paper further discusses
how these costs are mitigated, and how ATLAS is able to implement this
strategy without losing the ability to take advantage of the (improving!)
automatic schema evolution capabilities of underlying technology layers
when appropriate.
Implications of state representations for direct ROOT browability, and
current strategies for associating physics analysis views with such
state representations, are also described.
| Submitted on behalf of Collaboration (ex, BaBar, ATLAS) | ATLAS |
|---|
Primary authors
Dr
Arthur Schaffer
(LAL Orsay)
Dr
David Malon
(Argonne National Laboratory)
Dr
Kyle Cranmer
(Brookhaven National Laboratory)
Dr
Marcin Nowak
(Brookhaven National Laboratory)
Dr
Peter van Gemmeren
(Argonne National Laboratory)
Dr
Scott Snyder
(Brookhaven National Laboratory)
Dr
Sebastien Binet
(Lawrence Berkeley National Laboratory)
