AI, neural nets & generalization

Ok, so I can get neural nets to converge to solve a training set perfectly, but when I put other values, it always gives weird as fuck answers. In other words, my neural nets don't generalize (which is the point of AI), even if the training set is fuckhuge. How do I make them generalize? Some site on the internet told me to make the net smaller, but that just causes it to not even be able to solve the training set and become stuck on a local (nonzero) minimum.

I don't see how neural nets are any better than just making a program that creates random programs and changes them slightly until the result is appropriate, they don't generalize jack shit, it's the same as just returning
if(input==challenge1)return answer1;
if(input==challenge2)return answer2;
etc.

Yes, clearly when ancient, well-tested algorithms don't behave as documented, the problem is with the algorithms and not with your implementation.