27 September 2004 to 1 October 2004
Interlaken, Switzerland
Europe/Zurich timezone

Boosting the data logging rates in run 4 of the PHENIX experiment

28 Sep 2004, 10:00
1h
Coffee (Interlaken, Switzerland)

Coffee

Interlaken, Switzerland

Board: 21
poster Track 1 - Online Computing Poster Session 1

Speaker

Martin purschke

Description

With the improvements in CPU and disk speed over the past years, we were able to exceed the original design data logging rate of 40MB/s by a factor of 3 already for the Run 3 in 2002. For the Run 4 in 2003, we increased the raw disk logging capacity further to about 400MB/s. Another major improvement was the implementation of compressed data logging. The PHENIX raw data, after application of the standard data reduction techniques, were found to be further compressible by utilities like gzip by almost a factor of 2, and we defined a PHENIX standard of a compressed raw data format. The buffers that make up a raw data file consist of buffers that would get compressed and the resulting smaller data volume written out to disk. For a long time, this proved to be much too slow to be usable in the DAQ, until we could shift the compression to the event builder machines and so distributed the load over many fast CPU's. We also selected a different compression algorithm, LZO, which is about a factor of 4 faster than the "compress2" algorithm used internally in gzip. With the compression, the raw data volume shrinks to about 60% of the original size, boosting the original data rate before compression to more than 700MB/s. We will the present the techniques and architecture, and the impact this has had on the data taking in Run 4.

Primary author

C. Pinkenburg (BROOKHAVEN NATIONAL LABORATORY)

Presentation Materials