Major problem with time travel

Movies that feature time travel or even the general conception of the idea is like this: You retain your position in space and travel through time to appear at the same place in future or past. Here is my problem, did nobody else foresee this? If you go back in time and conserve your momentum( seems like a must) to appear at the same place, you will most likely end up in empty space or in a star because the earth, solar system, milky way galaxy fucking moves in space. With our current momentum we can't match Earth's motion through space and all travelers will die in vain.
Jumping in your DeLorean and appearing at the same place in future huh? No, you will drift in space with 85 miles per hour to eternity.

Perhaps the time travel machine in question would merely choose the position in space relative to its own.  For instance, in Primer, it always placed the person inside the machine, thus making it impossible for said person to appear in space or anything like that.  I do see where you're coming from, but to say that a person will retain their same position in space as they travel through time suggests that your location in space has any significance or may be determined without reference objects.  It would seem, though, that you would retain your momentum relative to objects even as your direction through time were reversed, thus requiring additional energy to change your relative momentum to stay within Earth.  This is all assuming your could travel through time, though.  The mere act of accelerating yourself to high enough a velocity to change your direction through time would require a literally infinite amount of energy, and traveling through wormholes that have been located at different points in time would make this entire point moot.