May 14 – 18, 2018
University of Wisconsin-Madison
America/Chicago timezone

INFN-T1 flooding report

May 17, 2018, 10:00 AM
Chamberlin Hall (University of Wisconsin-Madison)

Chamberlin Hall

University of Wisconsin-Madison

Madison, USA 43°4'25.8024''N 89°24'18.7776''W 43.073834, -89.405216
IT Facilities & Business Continuity IT facilities


Stefano Dal Pra (INFN)


On November 9 2017, a major flooding occurred in the computing rooms: this has turned into a down of all the services for a prolonged period of time.
In this talk we will go through all the issues we faced in order to recover the services in the quickest and most efficient way; we will analyze in detail the incident and all the steps made to recover the computing rooms, electrical power, network, storage and farming.
Moreover, we will discuss the hidden dependencies among services discovered during the recovery of the systems and will detail how we solved them.

Desired length 15

Primary authors

Presentation materials