Measurements of reconstructed jets and jet substructure offer opportunities to study fragmentation in a nuclear environment. Jet measurements in $p$+A collisions can provide crucial information for understanding the relationship between hard and soft processes in small collision systems. However, at RHIC this promise is complicated by the low jet energies and lack of hadronic calorimetry in the current experiments. In this talk, we report new results with reconstructed jets, including substructure measurements applying jet grooming techniques, in $p$+$p$ and $p$+Au collisions at a center of mass energy of 200 GeV using the PHENIX experiment. Jets are reconstructed from charged particle tracks and electromagnetic calorimeter clusters with the anti-kT algorithm. The measurements are unfolded for detector response using a multi-dimensional algorithm to extract both the cross section and jet substructure quantities in a self-consistent fashion. We report jet cross sections and substructure observables in 200 GeV $p$+$p$ collisions as well as the nuclear modification factor and substructure as a function of centrality for $p$+Au collisions. These measurements have implications for developing a quantitative understanding the modification of jets in heavier systems, such as Cu+Au and Au+Au collisions at RHIC.