The portal connecting the invisible and visible sectors is one of the most natural explanations of the dark world. However, the early-time dark matter production via the portal faces extremely stringent late-time constraints. To solve such tension, we construct the scalar-controlled kinetic mixing varying with the ultralight CP-even scalar's cosmological evolution. To realize this and eliminate the constant mixing, we couple the ultralight scalar within the mass range 10−33eV<<m0<<eV with the heavy doubly charged messengers and impose the Z2 symmetry under the dark charge conjugation. Via the varying mixing, the keV−MeV dark photon dark matter is produced through the early-time freeze-in when the scalar is misaligned from the origin and free from the late-time exclusions when the scalar does the damped oscillation and dynamically sets the kinetic mixing. We also find that the scalar-photon coupling emerges from the underlying physics, which changes the cosmological history and provides the experimental targets based on the fine-structure constant variation and the equivalence principle violation.