We consider toy models of holography arising from 3d Chern-Simons theory. In this context a duality to an ensemble average over 2d CFTs has been recently proposed. We put forward an alternative approach in which, rather than summing over bulk geometries, one gauges a one-form global symmetry of the bulk theory. This accomplishes two tasks: it ensures that the bulk theory has no global symmetries, as expected for a theory of quantum gravity, and it makes the partition function on spacetimes with boundaries coincide with that of a modular-invariant 2d CFT on the boundary. In particular, on wormhole geometries one finds a factorized answer for the partition function. In the case of non-Abelian Chern-Simons theories, the relevant one-form symmetry is non-invertible, and its gauging corresponds to the condensation of a Lagrangian anyon.