Interior structures of stellar objects might have small pressure anisotropy due to several reasons, including rotation and the presence of magnetic fields. Here, retaining the approximation of spherical symmetry, we study the possible role of small anisotropy in stellar interiors in theories of modified gravity, that are known to alter the hydrostatic equilibrium condition inside stars. We show how anisotropy may put lower and upper bounds on the modified gravity parameter depending on the polytropic equation of state, and determine them numerically. We also study the mass of stellar objects in these theories, assuming such equations of state, and find that the Chandrasekhar mass limit in white dwarf stars gets substantially modified compared to the isotropic case, even without assuming the presence of extreme magnetic fields. Effects of small pressure anisotropy on the Hydrogen burning limit in low mass stars are also briefly commented upon. It is shown that here the isotropic case can predict a theoretical lower bound on the scalar tensor parameter, in addition to a known upper bound.