In heavy ion collisions, the typical system size is large enough to be treated hydrodynamically but small enough for hydrodynamic fluctuations to be important and directly observable via event-by-event measurements. We present a general systematic formalism describing dynamics of fluctuations in an arbitrary relativistic hydrodynamic flow. We derive a deterministic evolution equation for the fluctuation modes which nontrivially matches the kinetic equation for phonons propagating on an arbitrary background including relativistic inertial and Coriolis forces due to acceleration and vorticity of the flow. We introduce a concept of confluent connection which takes into account the relativity of "equal time" in the definition of the equal-time correlator of fluctuation. We also describe the procedure of renormalization of short-distance singularities which eliminates cutoff dependence, allowing efficient numerical implementation of these equations.