Finding effective theories of modified gravity that can resolve cosmological singularities and avoid other physical pathologies such as ghost and gradient instabilities has turned out to be a rather difficult task. The concept of limiting curvature, where one bounds a finite number of curvature-invariant functions thanks to constraint equations, is a promising avenue in that direction, but its implementation has only led to mixed results. Cuscuton gravity, which can be defined as a special subclass of k-essence theory for instance, is a minimal modification of gravity since it does not introduce any new degree of freedom on a cosmological background. Importantly, it naturally incorporates the idea of limiting curvature as we will present. Accordingly, we show how models of cuscuton gravity possess non-singular cosmological solutions and how those appear stable at first sight. Yet, various subtleties arise in the perturbations such as apparent divergences, e.g., when the Hubble parameter crosses zero. We revisit the cosmological perturbations in various gauges and demonstrate that the stability results are robust even at those crossing points, although certain gauges are better suited to analyze the perturbations. We comment on the validity of the different gauges and present the behavior of the perturbations both in the ultraviolet and infrared. Finally, perturbations for a model of extended cuscuton (as a subclass of Horndeski theory) are also presented.