Chatbot ‘Eugene Goostman’ Becomes First Program to Pass Turing Test

The Turing test is a test of a machine’s ability to exhibit intelligent behaviour equivalent to, or indistinguishable from, that of a human. In the original illustrative example, a human judge engages in natural language conversations with a human and a machine designed to generate performance indistinguishable from that of a human being. All participants are separated from one another. If the judge cannot reliably tell the machine from the human, the machine is said to have passed the test. The test does not check the ability to give the correct answer to questions; it checks how closely the answer resembles typical human answers. The conversation is limited to a text-only channel such as a computer keyboard and screen so that the result is not dependent on the machine’s ability to render words into audio.

This weekend, a computer program convinced 33 percent of a panel of judges at the University Reading that it was a 13-year-old Ukrainian boy.

In what can be interpreted as brilliant in its deviousness or exploitative in its disregard for the spirit of Turing’s originally proposed test, Eugene’s creators kind of kluged their way to victory on this one, by having it pretend to be a 13-year-old, non-native-English-speaking Ukrainian. As Eugene’s creator Vladimir Veselov put it, “our main idea was that [Eugene] can claim that he knows anything, but his age also makes it perfectly reasonable that he doesn’t know everything.” Is it fair? Technically. But it’s not the least bit impressive, in a cognitive sense.