An economical proposal to utilize the Standard Model Higgs as the inflaton, the so called Higgs inflation, has recently drawn some attention. We apply the power-counting formalism for effective field theory to study the consistency of the Higgs inflation scenario. We show that from the Higgs scattering in both Einstein and Jordan frames of the theory, the unitarity bound on new physics is proximate to the Hubble scale during the inflation so the semi-classical approximation within the effective theory would not be valid. In covariant gauge, the unitarity problems come from the non-minimal Higgs self-couplings, which cannot be removed by field redefinitions. In unitary gauge, where only a single scalar can be redefined for canonical kinetic terms, the unitarity problems appear through non-minimal Higgs-gauge couplings.