Since the first detections of gravitational waves (GW) from merging binary black holes (BH), there has been a renewed interest in the possibility that at least some of these BHs could be primordial in origin. I will briefly discuss motivations for such primordial black holes (PBHs), formed from the collapse of large over-densities in the early Universe. I will then examine the on-going debate over whether LIGO and Virgo have indeed detected merging PBHs, and what we might learn if they did. Of course, the power of GWs in probing PBHs extends far beyond these 'direct' detections. I will also discuss more indirect probes, such as the stochastic GW background which may be produced along with PBH formation. The observation of GWs has thus opened up a new way to detect and study PBHs and to learn about the physics of the early Universe from which they formed.