Masturbation has various effects on your body; your brain's chemical balances (hence your mood,...), your speed of recovery, the development of new muscle tissue, etc.
There's is a strong streak of anti-clericalism in our society that makes many normal people into liberal reactionaries. Even the suggestion that masturbation isn't completely harmless fun will put frowns on people's faces and even start arguments where you're eventually written off as some sort of religious bigot (in fact, I am an atheist.)