Everyone knows that dealing with infinite sets in a too liberal manner eventually leads to contradictions, unless you do it very carefully (and even then, you might still get to get a slap on the back of your head, courtesy of Godel, of course).
Also, nobody said there can only be one interpretation of mathematics. If you feel like working in a modulo n space or whatever the fuck else you feel like so that discrete math doesn't lead to contradictions, go ahead. But once you get to calculus, you're kind of forced to use real numbers and infinitesimals. There's simply no way around defining what \sqrt{2} is.
If you want to just use mathematics to do real life things, I'd say you just use the existing working base. If you want to build Hilbert's program (and slowly become insane in the process of it), well, you'll have to find your own way around Godel. You cannot dismiss "1+1=2" just because of the ethnicity of the person who said it.