AI, neural nets & generalization

Ok, so I can get neural nets to converge to solve a training set perfectly, but when I put other values, it always gives weird as fuck answers. In other words, my neural nets don't generalize (which is the point of AI), even if the training set is fuckhuge. How do I make them generalize? Some site on the internet told me to make the net smaller, but that just causes it to not even be able to solve the training set and become stuck on a local (nonzero) minimum.

I don't see how neural nets are any better than just making a program that creates random programs and changes them slightly until the result is appropriate, they don't generalize jack shit, it's the same as just returning
if(input==challenge1)return answer1;
if(input==challenge2)return answer2;
etc.

You said "perfectly."  Errors in the code aside, it might be that you're overtraining your net, so that it always gets the right answer for the training set but is actually crazy.  What you want to do is split your "training" set 80/10/10, where 80% is used for actual training, 10 is used to see how well the trained data is working, and 10 is used to see the "real" performance of your net.  Because the first 10 is of course going to affect the training as well.  Got it?  There was an awesome free online course on Machine Learning that covered this from Stanford, taught by a guy with more communication skills than I have.  Anyway, google "overtraining" and you'll find something.  I think I need to google "overuse of quotes" now.