Major problem with time travel

Movies that feature time travel or even the general conception of the idea is like this: You retain your position in space and travel through time to appear at the same place in future or past. Here is my problem, did nobody else foresee this? If you go back in time and conserve your momentum( seems like a must) to appear at the same place, you will most likely end up in empty space or in a star because the earth, solar system, milky way galaxy fucking moves in space. With our current momentum we can't match Earth's motion through space and all travelers will die in vain.
Jumping in your DeLorean and appearing at the same place in future huh? No, you will drift in space with 85 miles per hour to eternity.

it's kind of like with newtonian gravity, it works fine to a certain degree but fails terribly at explaining the details. i think relativity is even more dangerous because einstein was such a "respected" person, nobody has the balls to question his weird assumptions and everybody tries to build his theories around those of einstein not leaving too much room for new thesis'.

as an example, if i said that passing the speed of light isn't a biggie and the only thing that will happen is that i cease to exist because the matter i'm made out won't be able to interact with its surroundings, somebody will point me to time dilation and tell me that i should reread it while the predictions of time dilation are just as weird if not even weirder.