Stellar halo of the Milky Way is believed to have formed through hierarchical mergers of small stellar systems such as dwarf galaxies. By studying the orbits and chemistry of very metal-poor halo stars, we can decipher the merger events of ancient galaxies, as well as their chemical properties. We applied a novel clustering method, StarGO, to the largest bright very metal-poor star catalog complied from the LAMOST DR3 VMP catalog and Gaia DR2. We found two significant substructures with retrograde orbits. Judging from their metallicities, these substructures are probably the remnants of tidally disrupted low-mass dwarf galaxies or ultra faint dwarf galaxies. One of the substructures, is confirmed to be dynamically associated with a well-studied strongly r-process element enhanced star (r-II star) and a CEMP-s star. Our finding favors the scenario that the progenitor is an ultra faint dwarf. High resolution spectroscopic studies will be undertaken to find more possible chemically peculiar stars in both of these substructures. It is the first attempt to study the birth environment of these stars from the relics of small dwarf galaxies.