but you cant teach a computer. You programming sucks. AI is shit. McCarthey is shit. Minsky is shit. MIT is shit. Go fuck yourself.
Name:
Anonymous2012-08-16 23:28
It's very risky to program superhuman AI to do something you think you want. Human values are extremely complex and fragile. Also, I bet my values would change if I had more time to think through them and resolve inconsistencies and accidents and weird things that result from running on an evolutionarily produced spaghetti-code kluge of a brain. Moreover, there are some serious difficulties to the problem of aggregating preferences from multiple people — see for example the impossibility results from the field of population ethics.
>if it is super intelligent, it will have its own purpose.
Well, it depends. "Intelligence" is a word that causes us to anthropomorphize machines that will be running entirely different mind architectures than we are, and we shouldn't assume anything about AIs on the basis of what we're used to humans doing. To know what an AI will do, you have to actually look at the math.
An AI is math: it does exactly what the math says it will do, though that math can have lots of flexibility for planning and knowledge gathering and so on. Right now it looks like there are some kinds of AIs you could build whose behavior would be unpredictable (e.g. a massive soup of machine learning algorithms, expert systems, brain-inspired processes, etc.), and some kinds of AIs you could build whose behavior would be somewhat more predictable (transparent Bayesian AIs that optimize a utility function, like AIXI except computationally tractable and with utility over world-states rather than a hijackable reward signal). An AI of the sort may be highly motivated to preserve its original goals (its utility function), for reasons explained in The Superintelligent Will.
Basically, the Singularity Institute wants to avoid the situation in which superhuman AIs' purposes are incompatible with our needs, because eventually humans will no longer be able to compete with beings whose "neurons" can communicate at light speed and whose brains can be as big as warehouses. Apes just aren't built to compete with that.
>Dr. Neil DeGrasse Tyson mentioned that if we found an intelligence that was 2% different from us in the direction that we are 2% different [genetically] from the Chimpansees, it would be so intelligent that we would look like beings with a very low intelligence.
Yes, exactly.
>How does your group see something of that nature evolving and how will we avoid going to war with it?
We'd like to avoid a war with superhuman machines, because humans would lose — and we'd lose more quickly than is depicted in, say, The Terminator. A movie like that is boring if there's no human resistance with an actual chance of winning, so they don't make movies where all humans die suddenly with no chance to resist because a worldwide AI did its own science and engineered an airborn, human-targeted supervirus with a near-perfect fatality rate.
The solution is to make sure that the first superhuman AIs are programmed with our goals, and for that we need to solve a particular set of math problems (outlined here), including both the math of safety-capable AI and the math of aggregating and extrapolating human preferences.
Obviously, lots more detail on our research page and in a forthcoming scholarly monograph on machine superintelligence from Nick Bostrom at Oxford University. Also see the singularity paper by leading philosopher of mind David Chalmers.