Prime Taradox

In the Twin Paradox, it's the twin that travels away from and back to earth that supposedly experience a shorter time for the journey, than the twin that remains on earth.

The though struck me; How do we know that it's the spaceship that travels away from and back to earth, and not the earth that travels, causing the spacetwin to appear older upon return?

>>2
From the frame of reference of the spaceship, wouldn't the spaceship be standing still, and the earth slow down and turn around? Not the other way around, or is there a way to distinguish this? If so, please enlighten me.