Efficient random number generation with high quality statistical properties and exact reproducibility of Monte Carlo simulation are important requirements in many areas of computational science. VecRNG is a package providing pseudo-random number generation (pRNG) in the context of a new library VecMath. This library bundles up several general-purpose mathematical utilities, data structures and algorithms having both SIMD and SIMT(GPUs) support based on VecCore. Several state-of-the-art RNG algorithms are implemented as kernels supporting parallel generation of random numbers in both scalar, vector and Cuda workflows. In this report, we will present design considerations, implementation details and computing performance of parallel pRNG engines on both CPU and GPU. Reproducibility of propagating multiple particles in parallel in HEP event simulation is demonstrated, using GeantV based examples, for both sequential and fine-grain track-level concurrent simulation workflows. Strategies for efficient uses of vectorized pRNG and non-overlapping streams of random number sequences in concurrent computing environments will be also discussed.