Going beyond the primary CMB, there is overwhelming evidence that measuring the late time effects on the CMB photons (secondaries) will provide new and valuable information for cosmological inference, in particular upon cross-correlating with large-scale structure surveys. It has been shown recently (arXiv:1812.03167) that the near-future CMB surveys and galaxy surveys will have the statistical power to make a first detection of the moving lens effect, a CMB modulation due to changing gravitational potentials as a result of cosmological structure moving transverse to the line of sight. We will describe the velocity reconstruction method we developed and discuss applications for the reconstructed transverse velocities. The large-scale velocity modes reconstructed with the moving lens effect can be used to cancel cosmic variance for the purpose of constraining local non-Gaussianity, and in principle, they can be used to measure quantities such as the absolute growth rate, which is useful for studying dark energy, modified gravity, and the effects of neutrino mass. Our study opens up a new and promising channel of investigation for near-future surveys.