%\documentclass[conference]{IEEEtran}
%\documentclass[journal]{IEEEtran}
\documentclass{jacow}
\usepackage{graphicx}

\begin{document}
\title{Data Acquisition and Protection System for a Multi-MHz Neutron Detector}

\author{
  M.~Drochner, H.~Kleines, R.~M\"oller and A.~Radulescu,\\
  Forschungszentrum J\"ulich / JCNS, J\"ulich, Germany \\
  M.~Delong, D.~McCormick,\\
  Reuter-Stokes / GE Energy, Twinsburg OH, USA
%\\
}

\maketitle

%\hspace{2cm}

\begin{abstract}
At the "KWS2" small angle scattering instrument at the "FRM-2" neutron
source at Garching, Germany a new 3He neutron detector was installed
and commissioned in 2015. It is built of 18 "8-pack" modules from
GE Power / Reuter-Stokes. Each of these modules has its own data
acquisition and slow control processor, using only Gigabit Ethernet as its
connection to the outside world.

We show how data acquisition, time synchronization and interaction
with the slow control system are laid out, as well as some first results
and performance data.
\end{abstract}

\section{Introduction}

To obtain sufficient angular coverage, the detector needs to fill
the instrument tube's profile as completely as possible. For
a modular, reusable design, the detector is made of modules
with different lengths, each holding eight gas tubes, called "8packs".

\begin{figure}[h]
\centering
\includegraphics[width=3.0in]{detphoto.eps}
\caption{
    The new detector in front of the instrument vessel.
}
\label{fig_picture}
\end{figure}

Fig.~\ref{fig_picture} shows the complete detector in front of the
instrument's vacuum vessel. There are 18 "8pack" modules of
three different lengths.

The concept of "8packs" is based on a detector design developed
previously at Oak Ridge National Lab, and has been presented at various
conferences (see eg. \cite{cooper8pack}). The
detector electronics have been redesigned completely by GE.

A major design objective for the new front end electronics was
to reduce the number of cables to the outside as much as is feasible.
Connections are needed for control, data transport, time
synchonization and power supply. This is all handled by
Gigabit Ethernet and its PoE (Power over Ethernet) and
PTP (Precision Time Protocol, IEEE1588) companion standards.

\begin{figure}[h]
\centering
\includegraphics[width=3.5in]{p9g.eps}
\caption{
    Components and interconnections of the new detector system.
    (picture from GE)
}
\label{fig_gescheme}
\end{figure}

Fig.~\ref{fig_gescheme} shows an overview of the electronics
components and cables. Two Gigabit Ethernet switches with
PoE and PTP capability are mounted directly at the detector
modules, within a special air cooled box. It was necessary to
use two rather than just one because the 18 detector modules
draw more electrical power than a single switch could
deliver via PoE.

Details are explained in the following chapters.

\section{Data acquisition}

Each "8pack" uses a XILINX "ZYNQ" FPGA-CPU combination as
its main control and processing component. This is a comparably
fast and energy-efficient embedded computer. The CPU operates
a Linux kernel which in particular supports powerful TCP/IP-based
networking.

For each detected neutron, the position along the tube is calculated
(using the measured pulse size at both ends of the tube, employing
a center-of-mass-like algorithm). This information, together with
information about the tube and a time stamp, is packed into
a so-called "event" structure and sent through a TCP/IP socket
connection to the outside data acquisition computer.

The data acquisition computer keeps a TCP connection to each of the
18 detector modules, reading event data as fast as possible to
avoid congestion inside the detector modules. A good reason for this
is that memory is much more plentiful and cheaper in a modern PC
than in an embedded computer.
Data are immediately sorted into a histogram, to support a live
display for the operator and help with statistics gathering and to
emulate the behaviour of other detectors. (There is an option
to store the original data stream for diagnostic purposes. Its
use is limited by the disk space available.)
The histogram is three-dimensional: the tube number and position
along the tube comprise the x/y coordinates. The third axis
consists of the time difference relative to an external
event, eg. the chopper's zero crossing.

The data histogramming program is coupled to a TACO server --
TACO \cite{taco} is used as measurement control middleware
currently at this instrument. A transition to TANGO \cite{tango}
is in preparation.


\section{Time synchronization}

To get accurate timing information relative to external events,
all "8packs" need to run with well synchronized system timers,
and external events need to be recorded with the same accuracy.
To support the latter, we are using a "time stamper" module
also offered by GE. This contains a base processor similar
to the one used in the "8packs". The inputs are general purpose TTL
signals, allowing interfacing to synchronization signals
from the chopper or other machine components such as sample
environments.

These front end components are connected to Gigabit Ethernet
switches supporting PTP. The time jitter achieved this way
is in the tens-of-nanoseconds order of magnitude which
is fully sufficient for neutron scattering and the neutron
speeds and interaction lengths which are typical here.

The root of the PTP time distribution is a so-called
"grandmaster" clock. For this, we are using a Meinberg
GPS controlled clock. The original intent was to use
GPS as a timing reference, but this was problematic
due to construction and regulation problems. We were not
allowed to install a GPS antenna outside the
instrument hall and to lay a cable through the wall.
Thus we are now operating the Meinberg clock in free-running
mode. The built in oscillator is temperature controlled
and very stable. We have not been aware of any drift in normal
operation.

\section{Slow control -- detector control}

The detector modules need some slow control such as when setting
of thresholds and high voltage, or monitoring
temperatures. For this, GE decided to use the EPICS \cite{epics}
control system. This information is accessible as process
variables using the standard EPICS "channel access" protocol.
Additionally, there are various parameters controlling
the calculation of neutron event positions from pulse
height information measured at both tube ends.
These parameters can be adjusted after calibration measurements
to make the data from different tubes fit together exactly.

\section{Slow control -- machine protection}

The detector is an expensive and critical instrument component.
This makes it necessary to protect it from technical faults
and operator errors as far as possible.
There are at least three possible problems to look at:
\begin{itemize}
\item Overheating due to insufficient air cooling
\item Neutron event rate higher than expected, possibly caused
  by being hit by the direct beam (beam stop fault or operator error)
\item Detector movement while high voltage is switched on
\end{itemize}

We are using a PLC to deal with this. The "8packs" send so-called
"heartbeat" network packets (UDP datagrams) to the PLC (which is equipped with
an Ethernet interface). These packets contain information
about temperature, count rates and high voltage state.
The PLC has access to motion control of the instrument's
mechanical axes. It can also switch off the power supplies
for the detector front end, and initiate the closing of the
instrument's beam shutter.
This way, it can handle the three problems mentioned above
adequately.

\section{Conclusion}

The detector's operation has been proven in tests and real measurements.
Data acquisition and control system interaction worked well.

In a test, we achieved a neutron event rate of about 5MHz at
a dead time of about 10 percent. This means that the detector
is among the fastest in the world used at neutron scattering
instruments.

\begin{thebibliography}{1}

\bibitem{cooper8pack}
R.~Cooper, DET1.6 The Multitube SANS Detector System
at Oak Ridge National Laboratory, SAS2009, Diamond Light Source,
Oxfordshire, UK, September 2009

\bibitem{taco}
TACO control system, http://www.esrf.eu/

\bibitem{tango}
TANGO control system, http://www.tango-controls.org/

\bibitem{epics}
Experimental Physics and Industrial Control System, http://www.aps.anl.gov/epics/

\end{thebibliography}

\end{document}
