The IceCube neutrino telescope at the South Pole has measured the atmospheric muon neutrino spectrum as a function of zenith angle and energy. We have performed a search for eV-scale sterile neutrinos by looking at distortion in those distributions. Such a sterile neutrino, motivated by the anomalies in short-baseline experiments, is expected to have a significant signature in the $\bar\nu_\mu$ survival probability due to matter induced resonant effects for energies of order 1 TeV. This effect makes this search unique and sensitive to small sterile mixings. In this talk, I will present the results of the IceCube sterile neutrino search using the a one year high energy sample and also our results obtained by looking at deviations of the standard oscillation pattern below 100 GeV from three years of DeepCore data.