Researchers dispute whether computer ‘Eugene Goostman’ passed Turing test

The University of Reading in England announced Sunday that a “historic milestone” had been reached for artificial intelligence, but not everyone is convinced.

‘Eugene’, a computer programme that simulates a 13 year old boy, was developed in Saint Petersburg, Russia. The development team includes Eugene’s creator Vladimir Veselov, who was born in Russia and now lives in the United States, and Ukrainian born Eugene Demchenko who now lives in Russia.

The Turing Test is based on 20th century mathematician and code-breaker Turing’s 1950 famous question and answer game, ‘Can Machines Think?’. The experiment investigates whether people can detect if they are talking to machines or humans. The event is particularly poignant as it took place on the 60th anniversary of Turing’s death, nearly six months after he was given a posthumous royal pardon.

If a computer is mistaken for a human more than 30% of the time during a series of five minute keyboard conversations it passes the test. No computer has ever achieved this, until now. Eugene managed to convince 33% of the human judges (30 judges took part – see more details below) that it was human.

This historic event was organised by the University’s School of Systems Engineering in partnership with RoboLaw, an EU-funded organisation examining the regulation of emerging robotic technologies.

Professor Kevin Warwick, a Visiting Professor at the University of Reading and Deputy Vice-Chancellor for Research at Coventry University, said: “In the field of Artificial Intelligence there is no more iconic and controversial milestone than the Turing Test, when a computer convinces a sufficient number of interrogators into believing that it is not a machine but rather is a human. It is fitting that such an important landmark has been reached at the Royal Society in London, the home of British Science and the scene of many great advances in human understanding over the centuries. This milestone will go down in history as one of the most exciting.

“Some will claim that the Test has already been passed. The words Turing Test have been applied to similar competitions around the world. However this event involved the most simultaneous comparison tests than ever before, was independently verified and, crucially, the conversations were unrestricted. A true Turing Test does not set the questions or topics prior to the conversations. We are therefore proud to declare that Alan Turing’s Test was passed for the first time on Saturday.

“Of course the Test has implications for society today. Having a computer that can trick a human into thinking that someone, or even something, is a person we trust is a wake-up call to cybercrime. The Turing Test is a vital tool for combatting that threat. It is important to understand more fully how online, real-time communication of this type can influence an individual human in such a way that they are fooled into believing something is true…when in fact it is not.”

Eugene was one of five supercomputers battling it for the Turing Test 2014 Prize. On winning the competition and achieving this historic milestone Vladimir Veselov said:

“I want to congratulate everyone who worked on Eugene Goostman. Our whole team is very excited with this result. It’s a remarkable achievement for us and we hope it boosts interest in artificial intelligence and chatbots. Special thanks to Professor Kevin Warwick and Dr Huma Shah for their effort in organising the event.

“Eugene was ‘born’ in 2001. Our main idea was that he can claim that he knows anything, but his age also makes it perfectly reasonable that he doesn’t know everything. We spent a lot of time developing a character with a believable personality. This year we improved the ‘dialog controller’ which makes the conversation far more human-like when compared to programs that just answer questions. Going forward we plan to make Eugene smarter and continue working on improving what we refer to as ‘conversation logic’.”

Among the judges tasked with separating the human and computer participants were the actor Robert Llewellyn, who played robot Kryten in the sci-fi comedy TV series Red Dwarf, and Lord Sharkey, who led the successful campaign for Alan Turing’s posthumous pardon last year.

Professor Warwick concluded: “Not long before he died on 7 June 1954 Alan Turing, himself a Fellow of the Royal Society, predicted that in time this test would be passed. It is difficult to conceive that he could possibly have imagined what computers of today, and the networking that links them, would be like.”

Further details about the Turing Test 2014:

Simultaneous tests as specified by Alan Turing
Each judge was involved in five parallel tests – so 10 conversations
30 judges took part
In total 300 conversations
In each five minutes a judge was communicating with both a human and a machine
Each of the five machines took part in 30 tests
To ensure accuracy of results, Test was independently adjudicated by Professor John Barnden, University of Birmingham, formerly head of British AI Society

Agencies/Canadajournal