In a linear Paul ion-trap, the structural transition from a 1-D linear chain of ions to a 2-D zigzag structure, known as the linear-zigzag transition, is a well-known behavior for crystals of laser cooled, trapped ions. Here we present the first studies of the linear-zigzag transition at ultracold temperatures following cooling to near the ground-state of motion. We characterize the transition using Raman sideband spectroscopy , revealing the shape of the effective potential near the critical point and thereby the nature of the transition. In an ideal linear Paul trap, the linear-zigzag transition is associated with the onset of a symmetric double-well potential that is indicative of its continuous nature . Experimentally in our setup, we observe a bias in the double-well potential near the critical point; we attribute this to small asymmetries in the ion trap . We also observe a shift in the critical point of approximately 0.5% from the value predicted in the pseudopotential approximation for the ion trap , consistent with the effect of micromotion on the ions in the radio-frequency trap. At a technical level, our spectroscopic measurements of the linear-zigzag transition are enabled by a high level of long-term trap stability over the duration of data acquisition. This work sets the stage for measurements of coherent effects near the critical point.