Nature has sprinkled black holes of various sizes throughout the universe, from stellar mass black holes in X-ray sources to supermassive black holes of billions of solar masses in quasars. Astronomers today are probing the spacetime near black holes using X-rays, and gravitational waves will open a different view in the near future. These tools give us an unprecedented opportunity to test ultra-strong-field general relativity, including the fundamental theorem of the uniqueness of the Kerr metric and Roger Penrose's cosmic censorship conjecture. Already, fascinating studies of spectral lines are showing the extreme gravitational lensing effects near black holes and allowing crude measurements of black hole spin. When the ESA-NASA gravitational wave detector LISA begins its observations in about 10 years, it will make measurements of dynamical spacetimes near black holes with an accuracy greater even than that which theoreticians can reach with their computations today. Most importantly, when gravitational waves from colliding black holes are detected we will have the first direct observations of black holes themselves.