Non-computability.

According to Roger Penrose, humans can perform non-computable feats, such as dealing with Gödel questions. He uses this as a foundation to claim that the human mind cannot be expressed in terms of classical processes, and as such must be party to the only other (known) game in town: Quantum Mechanics.

Now, I haven't had the patience to sit through all of his arguments yet, though I slowly make progress. My understanding is that a large part of his stance is that an algorithm cannot usefully deal with a Gödel question, or equivalently, with the halting problem, while a human can.

My objection to this is that such problems always demand a certain quality of response when asked of UTMs: failing to respond forever is not acceptable as correct, nor is providing any response other than one that yields a truth when taken in combination with the question. This much is fine, however, when it is time for the human to answer, he is permitted the liberty of rejecting the question on the grounds that it is inherently unanswerable.

Obviously I am interested in artificial intelligence, and also find his assertion to be simply a self-serving one with a contrived philosophical backdrop for foundation. If anyone knows of, or can think of, a more sophisticated argument than the one above (or expose my flaws in my assessment of it) I would like to hear it.

Apologies for bringing up a largely philosophical question, my only excuse is that I cannot trust any other board with the question.

>>74
and there's no question that a TM itself can be programmed to emulate or exhibit intelligence in one form or the other
I agree with this, because I've seen it done. The author of the 'Creatures' game wrote a book (Creation, Life and How to Make It.) about building the intelligence. It is a phenomenally elucidating read, and just about anyone can understand it. Going back to the point, his creatures certainly exhibit something I'd call intelligence, if not 'much' of it.

Our own self-reflecting behaviour is truly confusing to us making us think that it may be impossible to replicate this in a TM,
That's an interesting way to put it. I agree that it poses a hurdle, but we should realize that reflection is a necessary component of anything intelligent (I would argue it could be shown that anything that does not employ reflection cannot be intelligent--even if it could respond to all the same cases, in which case I'd call it an 'oracle' or 'demon' or something similar.) This implies a bit of knowledge about what intelligence is, but I think that can be taken care of simply by requiring it to be a wholly natural process.

but there is really no way to prove that without first understanding what exactly is this element of our intelligence, and if it even exists to begin with.
'Qualia' commonly receives the blame here. Some say they are merely an 'illusion'. Daniel Dennett has done quite a number on qualia--to the point where I have to agree with him when he says the term is so useless as to be unsalvageable. He has the guts to go deeper and consider what really is going on, instead of being dismissive (unlike most philosophers who would deny qualia.)

I don't think we'll find anything that is truly impossible to reproduce within computation. Anything we'd like think of that way will probably turn out to be ill-defined or completely fictitious.