The momentum distribution of quark-gluon plasma at the early stage of a relativistic heavy-ion collision is anisotropic and consequently the system, which is assumed to be weakly coupled, is unstable due to chromomagnetic plasma modes. We consider a high-energy parton which flies across such an unstable plasma, and the energy transfer between the parton and the medium is studied as an initial value problem. In the case of equilibrium plasmas, the well-known formula of collisional energy loss is reproduced. The unstable plasma case is much more complex, and the parton can lose or gain energy depending on the initial conditions. The extremely prolate and extremely oblate systems are considered as examples of unstable plasmas, and two classes of initial conditions are discussed. When the initial chromodynamic field is uncorrelated with the color state of the parton, it typically looses energy, and the magnitude of the energy loss is comparable to that in an equilibrium plasma of the same density. When the initial chromodynamic field is induced by the parton, it can be either accelerated or decelerated depending on the relative phase factor. With a correlated initial condition, the energy transfer grows exponentially in time and its magnitude can much exceed the absolute value of energy loss in an equilibrium plasma. The energy transfer is also strongly directionally dependent. Consequences of our findings for the phenomenology of jet quenching in relativistic heavy-ion collisions are briefly discussed.