14-18 October 2013
Amsterdam, Beurs van Berlage
Europe/Amsterdam timezone

WAN Data Movement Architectures at US-LHC Tier-1s

17 Oct 2013, 11:46
Veilingzaal (Amsterdam, Beurs van Berlage)


Amsterdam, Beurs van Berlage

Oral presentation to parallel session Facilities, Production Infrastructures, Networking and Collaborative Tools Facilities, Infrastructures, Networking and Collaborative Tools


Mr Phil Demar (Fermilab)


LHC networking has always been defined by high volume data movement requirements in both LAN and WAN. LAN network demands can typically be met fairly easily with high performance data center switches, albeit at high cost. LHC WAN data movement, on the other hand, presents a more complicated and difficult set of challenges. Typically, there are three high-level issues a high traffic volume LHC site needs to deal with in providing a quality LHC WAN service: - Ensuring sufficient bandwidth capacity for the LHC data - Protecting the site’s other WAN traffic from being negatively impacted by LHC traffic flows - Contending with the site’s perimeter security policies and mechanisms The emergence of alternate network paths specifically for LHC data movement has provided a means for many LHC sites to appropriately deal with the first two issues. The LHCOPN and LHCONE are examples of physical and virtual network infrastructure respectively that enable sites to direct their LHC WAN traffic over adequately provisioned, isolated network paths. However, the site must still deal with its local security policies to move that traffic through its perimeter. Historically, the firewalls and security tools used to implement local security policies have not been capable of keeping up with LHC WAN traffic loads. This problem normally necessitates use of perimeter bypass mechanisms. ESnet has pioneered in the development of the Science DMZ, a general architecture for separating high impact science data flows from a site’s normal routed internet traffic. Like any architecture, implementation varies according to circumstances and conditions. This presentation will discuss the concept of the science DMZ architecture, with a focus on implementation of that concept at the two US Tier-1 facilities, Fermilab (CMS) and Brookhaven National Laboratory (Atlas). The talk will discuss how each US Tier-1 has structured and configured its network perimeter infrastructure to meet the demands of its LHC WAN data movement, while still maintaining a secure network perimeter consistent with its overall security policies. Particular emphasis will be given to deployment of 100GE WAN technology on the site perimeter. Both US Tier-1 facilities are currently in the process of deploying 100GE support for their LHC data movement, and implementation details will be covered.

Primary author

Mr Phil Demar (Fermilab)


Mr Scott Bradley (Brookhaven National Laboratory)

Presentation Materials