Time paradox

lets take an example.

A man travels back in time and does something insignificant, thus "butterfly" effect goes haywire and he winds up never having been born. BUT if he was never born how could he have gone back in time and caused said time paradox to stop him being born?

If you were to go back in time, two things could happen.. if there is one universe and one timeline, then you cannot affect the past.. since your time travel trip is already part of it.. and anything you do infact is part of the past already and creates your future, and if you try to kill say.. your father, before you where born.. it would not work.. you would probably die before you got the chance.. or just never successfully kill him, the attempt would always get botched, infact, you might even be the reason you were first born by making you mother and father meet or something.

If there are an infinite amount of parallel universes in a sort of "multiverse", then going to the past and affecting it would place you in a new universe with this new altered existence.