The discovery of the Higgs boson by ATLAS and CMS has opened a new frontier of exciting measurements in Particle Physics. The tau lepton, a peculiar cousin of the electron, offers the opportunity to study a new type of force: the Yukawa interaction. However, detecting the tau lepton is experimentally very challenging as it predominantly decays into pions and an undetectable neutrino. In this talk, I will discuss the strategy employed by the ATLAS experiment to detect hadronic tau decays and the modern machine learning algorithms employed to distinguish them from QCD jets. I will present the results of the Higgs boson coupling to the tau, the most precise Higgs boson measurements in the fermionic sector. I will then turn to the tau lepton itself, and present the first result from ATLAS on the anomalous magnetic moment, a quantity that is yet to be measured. Finally I will discuss the long term prospects of this measurement program at the LHC and beyond.