Speaker
Dr
Wenji Wu
(Fermi National Accelerator Laboratory)
Description
Fermilab is the US-CMS Tier-1 Centre, as well as the main data centre for several other large-scale research collaborations. As a consequence, there is a continual need to monitor and analyse large-scale data movement between Fermilab and collaboration sites for a variety of purposes, including network capacity planning and performance troubleshooting. To meet this need, Fermilab designed and implemented a network traffic characterization system for our large-scale bulk data transfers. The original version of the system simply analysed flow data sequentially on a conventional multicore system. That design had two significant limitations. First, there was an appreciable delay in the analysis. While the results were still useful for network characterization studies and capacity planning purposes, they were not available in near real-time. Such a capability would open up more operationally-oriented uses. Second, the system would not scale well to 40GE and 100GE network environments, due to the sequential analysis.
Fermilab is currently developing an enhanced system which remedies those limitations. The system consists of two major components, a flow-data collection component and a data analysis engine based on GPU (graphic processing unit) technology. Our system exploits the data parallelism that exists within network flow data to provide accurate monitoring and near real-time analysis of bulk data movement. The system is designed to work in 40GE/100GE network environments. In this paper, we discuss the architecture and design of our system, including some of the applications available. Results from analysis on production traffic will be incorporated into the presentation.
Primary author
Dr
Wenji Wu
(Fermi National Accelerator Laboratory)
Co-author
Mr
Phil Demar
(Fermilab)
