Speaker
Description
Summary 500 words
The KM3NeT off-shore readout and control system is strongly constrained by the submarine telescope major characteristics. The physical distribution of the detector calls for a highly distributed, embedded data acquisition system. It has to configure, monitor and collect data from about 12900 detecting nodes spread over 322 vertical lines. A reliable low-power and compact off-shore processor node has been designed to comply with the very restrictive electronics access for maintenance and the tight mechanical housing. It is located inside a Digital Optical Module (DOM) housing thirty-one 3” PMTs, each PMT base delivers a Time over Threshold (TOT) signal. The DOM instrumentation and configuration are carried over local standard I2C and SPI links. It is connected to the shore via a gigabit Ethernet link. The processor node collects and time stamps the 31 TOT signals, encapsulates the acquired data in Ethernet frames sent to shore. It configures the DOM acquisition and reads out instruments upon shore station remote control. A Readout System on Chip (RSOC) design based on Commodity Off-The-Shelf (COTS) hardware and software components is selected to fulfill the requirements. The RSOC is implemented in a high density FPGA. It integrates a processor, the PMTs data readout, I2C and SPI IPs and a customized gigabit Ethernet link. The dynamic and non-volatile memories still need to be implemented as separate components. The processor runs the Real Time Operating System (RTOS) VxWorks from Wind River. The selected device for the preproduction model is from the Xilinx Virtex5-FX family. The design makes use of the embedded PowerPC 440 core, the Tri-mode Ethernet Media Access Controller (TEMAC) core, the LVDS, SERDES and GTX Rocket I/O. A flash memory includes both the Xilinx configuration and the first processor boot code which downloads the RTOS image through Ethernet.
The synchronization of the distributed KM3NeT off-shore nodes is achieved by the distribution of a clock signal from the on-shore electronics up to 100 km away from the seabed network of detectors. The distributed clock signal is recovered off-shore from the 8bit/10bit encoded 1.25 Gbps serial link used for the 1000BASE-X Ethernet communication to shore. The GTX provides the means to align the serial data flow and the recovered parallel clock based on the detection of special coma characters. However the automatic coma alignment causes the latency of the serial link to vary after each reset or loss of lock. Non standard parameterizations of the GTX enable to overcome this issue, leading to known or fixed latency allowing for clock distribution with sub-nanosecond precision. The synchronization of the detecting nodes requires the ability to send from shore specific commands, typically a “start counter”, received off-shore on a deterministic edge of the Ethernet recovered parallel clock. This design manages to send and receive such commands without hampering the standard Ethernet protocol for higher level communications.
The RSOC implements a “store and forward” data acquisition model. The RSOC software makes use of The Internet Communication Engine (Ice) from ZeroC to ease the control from shore of the 12900 detecting nodes.
