It is known that there is no upper (extremal) limit on the spin of rotating black holes in dimensions D > 5, when rotation takes place on only one plane. We first show that the shape of an ultra-spinning black hole becomes highly flattened ('pancaked') along the plane of rotation, and then argue that, indeed, for high enough rotation these black holes will likely become unstable. We discuss several possible modes for the decay. We also discuss the possible relevance of this instability for black holes in brane-world models with large extra dimensions. Based on work in collaboration with Rob Myers.