We discuss the implications for the Yukawa couplings of the Standard Model of the measurement of quark, charged lepton and neutrino masses, mixings and CP violation. The resultant structure is suggestive of a (broken) family symmetry and we consider what is the maximal symmetry which, when combined with a Grand Unified symmetry and the seesaw mechanism, is consistent with both the quark and lepton sectors. This naturally accommodates large mixing angles in the lepton sector and leads to new texture zero predictions for the CHOOZ angle and CP violating phases. We construct a supersymmetric model of masses and mixings based on SO(10) together with an SU(3) family symmetry. In addition to giving a good description of the fermion sector it limits flavour changing processes through constraints on the spectrum of squarks and sleptons. Spontaneous breaking of CP is easily implemented via the flavon sector and gives a simple way of avoiding the SUSY CP problem.