Speaker
Description
TCP has been widely used in readout systems. SiTCP is hardware-based TCP stack for Gigabit Ethernet, it realizes direct access and transfer of the data up to 949 Mbps in the memory of FPGA by utilizing TCP communication. The data rate multiplies with the development of pixel detectors for smaller pixels and higher frame-rates. The existing GbE design is no longer satisfied the requirement of transmission bandwidth. This paper describes a multi-threaded TCP hardware stack based on SiTCP for 10 GbE implementing on a single FPGA. The throughput from FPGA to PC is at the upper limits of 10 GbE.
Summary
1、Background：the introduction of pixel detector.
The hybrid pixel detector consists of a silicon sensor and a readout chip which is bump-bonded to the sensor with Indium. The sensor contains an array of 288 x 208 pixels while each pixel measures 150 μm×150 μm. Each pixel of the readout chip comprises a preamplifier, a discriminator and a counter. Aiming at X-ray imaging, pixel detector works in the single photon counting mode, the counting depth of every pixel is 20 bits. Therefore, in the 1.2 kHz frame rate, the throughput of each daisy chain (6 sensors) is around 8.63 Gbps. In order to achieve the purpose of real-time displaying data, we have to use 10 Gbps Ethernet.
2、The multi-threaded TCP hardware stack:
The Transmission control protocol (TCP) and Ethernet have been widely used in readout systems. Because of the functions of retransmission and flow control, it guarantees data reliability and stability. The SiTCP is the standard IP blocks, which provides FIFO (First In, First Out) interface that is convenient to connect with users’ logic. The PHY layer can interact directly with the SiTCP via the Gigabit Media Independent Interface (GMII). First, we convert GMII to 10 Gigabit Media Independent Interface (XGMII) via asynchronous FIFOs, and use a simple hub module to connect multi XGMII. If the package comes from the up-port, this hub module will broadcasts to the down-ports. If the packages come from the down-ports, this module will sends packages in roll polling mode to the up-port with the Inter-Packet Gap (IPG). Second, we overclock the SiTCP up to 250MHz beyond its specification of 125MHz in order to run faster. The timing is carefully examined and the signals crossing clock domains are carefully handled to avoid metastability.
In the project of silicon pixel detector, TCP is used to transmit data and UDP is used for interaction between the host computer and the readout. A kind of control bus protocol interface is provided, which helps the host computer to read and write the specific register directly through UDP protocol.
Transfer performance was confirmed with the readout circuit. The multi-threaded TCP hardware stack with five SiTCPs transfers data to PC at a total of 9.2 Gbps, corresponding to the limit of TCP using 10 GbE calculated with overheads, such as protocol headers.
References: T. Uchida, ”Hardware-Based TCP Processor for Gigabit Ethernet,” IEEE Trans. Nucl. Sci., vol. 55, no. 3, pp. 1631-1637, June 2008
