Although Cosmic Microwave Background and Large Scale Structure probe the largest scales of our universe with ever increasing precision, our knowledge about the smaller scales is still very limited other than the bounds on Primordial Black Holes(PBHs). We show that the statistical properties of the small scale quantum fluctuations can be probed via the stochastic gravitational wave background, which is induced as the scalar modes re-enter the horizon. We found that even if scalar curvature fluctuations have a subdominant non-Gaussian component, these non-Gaussian perturbations can source a dominant portion of the induced GWs. Moreover, the GWs sourced by non-Gaussian scalar fluctuations peaks at a higher frequency and this can result in distinctive observational signatures. If the induced GW background is detected, but not the signatures arising from the non-Gaussian component, this translates into stringent bounds on non-Gaussianity depending on the amplitude of the GW signal. The induced GWs can also give valuable information about the accretion and merging histories of the black holes as they are associated with the formation properties of the PBH.